149 lines
5.9 KiB
TeX
149 lines
5.9 KiB
TeX
\chapter{Evaluation}
|
|
|
|
In this chapter we will present the results of running each of the proposals discussed in this thesis on large-scale JavaScript projects.
|
|
|
|
To evaluate this tool on existing JavaScript codebases, we have collected JavaScript projects from Github containing many or large JavaScript files.
|
|
|
|
Each case study was evaluated by running this tool on every \texttt{.js}-file in the repository,
|
|
and then collecting the number of matches found in total,
|
|
and how many files were successfully searched.
|
|
Evaluating if the transformation was correct is done by manually sampling output files,
|
|
and verifying that it passes through Babel Generate~\cite{BabelGenerate}
|
|
without errors.
|
|
|
|
We describe below our results and observations on using our tool on the codebases of various large-scale projects that use JavaScript.
|
|
|
|
\textbf{Next.js}~\cite{NEXT.JS} is one of the largest projects on the web. It is used with React~\cite{React} to enable feature such as server-side rendering and static site generation.
|
|
|
|
\begin{table}[H]
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files processed\\
|
|
\hline \hline
|
|
``Pipeline'' & 242079 & 1912 & 3340 \\
|
|
\hline
|
|
``Do Expression'' & 229 & 37 & 3340 \\
|
|
\hline
|
|
Await to Promise & 8 & 7 & 3340 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with Next.js source code}
|
|
\label{fig:evalNextJS}
|
|
\end{table}
|
|
|
|
\textbf{Three.js}~\cite{ThreeJS} is a library for 3D rendering in JavaScript.
|
|
It is written purely in JavaScript and uses GPU for 3D calculations.
|
|
|
|
\begin{table}[H]
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched \\
|
|
\hline \hline
|
|
Pipeline & 84803 & 1117 & 1384 \\
|
|
\hline
|
|
``Do Expression'' & 248 & 36 & 1384 \\
|
|
\hline
|
|
Await to Promise & 13 & 7 & 1384 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with Three.js source code}
|
|
\label{fig:evalThreeJS}
|
|
\end{table}
|
|
|
|
\textbf{React}~\cite{React} is a graphical user interface library for JavaScript, which facilitates the creation of user interfaces for both web and native platforms. React is based upon splitting a user interface into components for simple development. It is currently one of the most popular libraries for creating web apps.
|
|
|
|
\begin{table}[H]
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched\\
|
|
\hline \hline
|
|
``Pipeline'' & 16353 & 1266 & 2051 \\
|
|
\hline
|
|
``Do Expression'' & 0 & 0 & 2051 \\
|
|
\hline
|
|
Await to Promise & 30 & 13 & 2051 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with React source code}
|
|
\label{fig:evalReact}
|
|
\end{table}
|
|
|
|
|
|
\textbf{Bootstrap}~\cite{Bootstrap} is a front-end framework used for creating responsive and mobile-first websites, and it comes with a variety of built-in components. This library is a good evaluation point for this thesis as it is written in ``vanilla'' JavaScript.
|
|
|
|
\begin{table}[H]
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched\\
|
|
\hline \hline
|
|
"``Pipeline'' & 13794 & 109 & 115 \\
|
|
\hline
|
|
``Do Expression'' & 0 & 0 & 115 \\
|
|
\hline
|
|
Await to Promise & 0 & 0 & 115 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with Bootstrap source code}
|
|
\label{fig:evalBootstrap}
|
|
\end{table}
|
|
|
|
|
|
\textbf{Atom}~\cite{Atom} is a text editor developed in JavaScript.
|
|
|
|
\begin{table}[H]
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched\\
|
|
\hline \hline
|
|
``Pipeline'' & 40606 & 361 & 401 \\
|
|
\hline
|
|
``Do Expression'' & 3 & 3 & 401 \\
|
|
\hline
|
|
Await to Promise & 12 & 7 & 401 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with Atom source code}
|
|
\label{fig:evalAtom}
|
|
\end{table}
|
|
|
|
The ``Pipeline'' proposal is applicable to most files: the reason for this is that call expressions are widely used when writing JavaScript code. Our tool found matches in most files that Babel~\cite{BabelParser} managed to parse, and with manual evaluation transformations were performed correctly.
|
|
|
|
The ``Do Expression'' proposal is not as ``applicable'' as the ``Pipeline'' proposal: this means that the amount of transformed code this specification in \DSL{} will be able to perform is expected and proven to be lower.
|
|
This is because the proposal introduces an entirely new way of writing
|
|
expression-oriented code in JavaScript.
|
|
If the code has not used the current way of writing
|
|
expression-oriented in JavaScript,
|
|
\DSL{} is limited in the amount of transformations it can perform.
|
|
Nevertheless, our tool is able to identify matches where it is applicable, and by manual verification transformations are correct.
|
|
|
|
The imaginary ``Await to promise'' proposal also has an ``expected'' number of matches; however, we do not evaluate this proposal since it is not an official TC39 proposal.
|
|
|
|
|
|
Our tool demonstrates its capability to perform searches on large codebases,
|
|
to identify applicable code for proposals,
|
|
and to transform the code.
|
|
As can be seen from the tables above,
|
|
some of the proposals found zero matches
|
|
when evaluated on some of these codebases.
|
|
This is due to the fact that the developers of these projects
|
|
have not used the language construct the proposal is targeting.
|
|
Because of this,
|
|
no transformations can be performed.
|
|
This is especially apparent with the ``Do Expression`` proposal,
|
|
but also with the ``Await to Promise'' imaginary proposal.
|
|
This means that our tool's ability to perform transformations
|
|
depends on how widespread the adoption of the language construct targeted in a proposal is.
|
|
We can hypothesize that the amount of matches reflects the ``impact''
|
|
that design decisions made by the TC39 committee might have on established JavaScript projects and codebases.
|
|
|
|
We give examples of some of the transformations performed on these codebases in Appendix~\ref{appendix:b}.
|