128 lines
No EOL
5.7 KiB
TeX
128 lines
No EOL
5.7 KiB
TeX
\chapter{Evaluation}
|
|
|
|
In this chapter we will present the results of running each of the proposals discussed in this thesis on large scale JavaScript projects.
|
|
|
|
\section{Real Life source code}
|
|
|
|
To evaluate this tool on real life JavaScript code, we have collected some JavaScript projects from Github containing many or large JavaScript files. Every JavaScript file within the project is transformed with our tool, and we will evaluate it based upon the amount of matches discovered, as well as manual checking that the transformation resulted in correct output code.
|
|
|
|
Each case study was evaluated by running this tool on every .js file in the repository, then collecting the number of matches found in total and how many files were successfully searched. Evaluating if the transformation was correct is done by manually sampling output files, and verifying that it passes through Babel Generate~\cite{BabelGenerate} without error.
|
|
|
|
``Pipeline''~\cite{Pipeline} is applicable to most files, as the concept it touches (function calls) is widely used when writing JavaScript code. Our tool found matches in almost all files that Babel~\cite{BabelParser} managed to parse, and the transformed files are correct.
|
|
|
|
The ``Do Expression'' proposal~\cite{Proposal:DoProposal} is not as applicable as the "Pipeline" proposal, as it does not re-define a concept that is used quite as frequently as call expressions. This means the amount of transformed code this specification in \DSL{} will be able to perform is expected and proven to be lower. This is due to the ``Do Expression'' proposal introducing an entirely new way to write expression-oriented code in JavaScript. If the code has not used the current way of writing in an expression-oriented style in JavaScript, \DSL{} is limited in the amount of transformations it can perform. Nevertheless, matches are discovered and transformed correctly.
|
|
|
|
|
|
The imaginary ``Await to promise'' proposal also has an expected number of matches, but this evaluation proposal is not meant to be real life representative, and was only created to evaluate the expressiveness of our tool.
|
|
|
|
|
|
|
|
|
|
\textbf{Next.js}~\cite{NEXT.JS} is one of the largest projects on the web. It is used with React~\cite{React} to enable feature such as server-sire rendering and static site generation.
|
|
|
|
\begin{table}[H]
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files processed\\
|
|
\hline \hline
|
|
``Pipeline'' & 242079 & 1912 & 3340 \\
|
|
\hline
|
|
``Do Expression'' & 229 & 37 & 3340 \\
|
|
\hline
|
|
Await to Promise & 143 & 75 & 3340 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with Next.js source code}
|
|
\label{fig:evalNextJS}
|
|
\end{table}
|
|
|
|
|
|
|
|
|
|
|
|
\textbf{Three.js}~\cite{ThreeJS} is a library for 3D rendering in JavaScript. It is written purely in JavaScript and uses GPU for 3D calculations. It being a popular JavaScript library, and being written in mostly pure JavaScript makes it a good case study for our tool. It currently sits at over 1 million downloads weekly.
|
|
|
|
\begin{table}[H]
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched \\
|
|
\hline \hline
|
|
Pipeline & 84803 & 1117 & 1384 \\
|
|
\hline
|
|
``Do Expression'' & 248 & 36 & 1384 \\
|
|
\hline
|
|
Await to Promise & 13 & 7 & 1384 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with Three.js source code}
|
|
\label{fig:evalThreeJS}
|
|
\end{table}
|
|
|
|
\textbf{React}~\cite{React} is a graphical user interface library for JavaScript, it facilitates the creation of user interfaces for both web and native platforms. React is based upon splitting a user interface into components for simple development. It is currently one of the most popular libraries for creating web apps and has over 223000 stars on Github.
|
|
|
|
\begin{table}[H]
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched\\
|
|
\hline \hline
|
|
``Pipeline'' & 16353 & 1266 & 2051 \\
|
|
\hline
|
|
``Do Expression'' & 0 & 0 & 2051 \\
|
|
\hline
|
|
Await to Promise & 30 & 13 & 2051 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with React source code}
|
|
\label{fig:evalReact}
|
|
\end{table}
|
|
|
|
|
|
\textbf{Bootstrap}~\cite{Bootstrap} is a front-end framework used for creating responsive and mobile-first websites, it comes with a variety of built-in components, as well as a built in styling. This styling is also customizable using CSS. This library is a good evaluation point for this thesis as it is written in pure JavaScript and is used by millions of developers.
|
|
|
|
\begin{table}[H]
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched\\
|
|
\hline \hline
|
|
"``Pipeline'' & 13794 & 109 & 115 \\
|
|
\hline
|
|
``Do Expression'' & 0 & 0 & 115 \\
|
|
\hline
|
|
Await to Promise & 0 & 0 & 115 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with Bootstrap source code}
|
|
\label{fig:evalBootstrap}
|
|
\end{table}
|
|
|
|
|
|
|
|
\textbf{Atom}~\cite{Atom} is a text editor made in JavaScript using the Electron framework. It was created to give a very minimal and modular text editor. It was bought by Microsoft, and later discontinued in favor for Visual Studio Code.
|
|
|
|
\begin{table}[H]
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched\\
|
|
\hline \hline
|
|
``Pipeline'' & 40606 & 361 & 401 \\
|
|
\hline
|
|
``Do Expression'' & 3 & 3 & 401 \\
|
|
\hline
|
|
Await to Promise & 12 & 7 & 401 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with Atom source code}
|
|
\label{fig:evalAtom}
|
|
\end{table}
|
|
|
|
See Appendix~\ref{appendix:b} for some examples of the transformations discussed in this section. |