136 lines
No EOL
6.3 KiB
TeX
136 lines
No EOL
6.3 KiB
TeX
\chapter{Evaluation}
|
|
|
|
In this chapter we will discuss how we evaluated \DSL and its related tools. This chapter will include some testing of the tool on demo code snippets, as well as running each of the proposals discussed in this thesis on some large scale JavaScript projects.
|
|
|
|
\section*{Testing on code}
|
|
|
|
In this section, we will showcase some synthetic transformations applied to code made to fit each of the definitions discussed in this thesis.
|
|
|
|
|
|
The pipeline proposal is meant to merge the readability of chaining with the practicality of deeply nested calls. This deep nesting can be seen in the input code Listing \ref{ex:deepNestingEval}. The resulting code can be seen in Listing \ref{ex:transformedDeepNestingEval}.
|
|
|
|
\begin{lstlisting}[language={JavaScript}, label={ex:deepNestingEval}]
|
|
// Original JavaScript
|
|
a(a(a(a(a(a(a(b)))))));
|
|
c(c(c(c(c(d, b), b), b), b), b);
|
|
\end{lstlisting}
|
|
\begin{lstlisting}[language={JavaScript}, label={ex:transformedDeepNestingEval}]
|
|
// Transformed
|
|
b |> a(%) |> a(%) |> a(%) |> a(%) |> a(%) |> a(%) |> a(%);
|
|
d |> c(%, b) |> c(%, b) |> c(%, b) |> c(%, b) |> c(%, b);
|
|
\end{lstlisting}
|
|
|
|
\section{Real Life source code}
|
|
|
|
In order to perform actual large scale trial of this program, we have collected some github projects containing many or large JavaScript files. Every JS file within the project is then passed through the entire tool, and we will evaluate it based upon the amount of matches discovered, as well as manual checking that the transformation resulted in correct code on the matches.
|
|
|
|
Each case study was evaluated by running this tool on every .js file in the repository, then collecting the number of matches found in total and how many files were successfully searched. Evaluating if the transformation was correct is done by manually sampling output files, and verifying that it passes through Babel Generate \cite{BabelGenerate} without error. You can see some highlighted transformations in Listing \textbf{INSERT FIGURE HERE}.
|
|
|
|
|
|
"Pipeline"\cite{Pipeline} is clearly very applicable to most files, as it is really only looking for function calls. This is by far the best result, and it found matches in almost all files that Babel \cite{BabelParser} managed to parse.
|
|
|
|
The Do proposal \cite{Proposal:DoProposal} is expected to not find many matches, as code that has not been written in expression-oriented programming style will not produce many matches.
|
|
|
|
Await to promise also has an expected number of matches, but this evaluation proposal is not meant to be real life representative. As it is limited to functions containing only a single await statement and that statement has to be a \texttt{VariableDeclaration}.
|
|
|
|
When the amount of files searched is different for each of the proposals, it means either a transformation on that file, or generation of that file failed for some reason. This is probably due to bugs in the transform algorithm that creates this discrepancy.
|
|
|
|
\paragraph*{Next.js}\cite{NEXT.JS} is one of the largest projects related to JavaScript. It is owned by Vercel and is used as a development platform for the web.
|
|
|
|
\paragraph*{Three.js}\cite{ThreeJS} is a library for 3D rendering in JavaScript. It is written purely in JavaScript and uses GPU for 3D calculations. It being a popular JavaScript library, and being written in mostly pure JavaScript makes it a good case study for our tool. It currently sits at over 1 million downloads weekly.
|
|
|
|
\paragraph*{React}\cite{React} is a graphical user interface library for JavaScript, it facilitates the creation of user interfaces for both web and native platforms. React is based upon splitting a user interface into components for simple development. It is currently one of the most popular libraries for creating web apps and has over 223000 stars on Github.
|
|
|
|
\paragraph*{Bootstrap}\cite{Bootstrap} is a front-end framework used for creating responsive and mobile-first websites, it comes with a variety of built-in components, as well as a built in styling. This styling is also customizable using CSS. This library is a good evaluation point for this thesis as it is written in pure JavaScript and is used by millions of developers.
|
|
|
|
\paragraph*{Atom}\cite{Atom} is a text editor made in JavaScript using the Electron framework. It was created to give a very minimal and modular text editor. It was bought by Microsoft, and later discontinued in favor for Visual Studio Code.
|
|
|
|
\begin{figure}
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched\\
|
|
\hline \hline
|
|
Pipeline & 55787 & 1327 & 2331 \\
|
|
\hline
|
|
Do & 131 & 74 & 2331 \\
|
|
\hline
|
|
Await to Promise & 43 & 38 & 2331 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with Next.js source code}
|
|
\label{fig:evalNextJS}
|
|
\end{figure}
|
|
|
|
\begin{figure}
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched \\
|
|
\hline \hline
|
|
Pipeline & 85050 & 1119 & 1262\\
|
|
\hline
|
|
Do & 278 & 55 & 1385 \\
|
|
\hline
|
|
Await to Promise & 125 & 80 & 1262 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with Three.js source code}
|
|
\label{fig:evalThreeJS}
|
|
\end{figure}
|
|
|
|
\begin{figure}
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched\\
|
|
\hline \hline
|
|
Pipeline & 16353 & 1266 & 2051 \\
|
|
\hline
|
|
Do & 79 & 60 & 2051 \\
|
|
\hline
|
|
Await to Promise & 104 & 88 & 2051 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with React source code}
|
|
\label{fig:evalReact}
|
|
\end{figure}
|
|
|
|
\begin{figure}
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched\\
|
|
\hline \hline
|
|
Pipeline & 13794 & 109 & 115 \\
|
|
\hline
|
|
Do & 13 & 8 & 115 \\
|
|
\hline
|
|
Await to Promise & 0 & 0 & 115 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with Bootstrap source code}
|
|
\label{fig:evalBootstrap}
|
|
\end{figure}
|
|
|
|
\begin{figure}
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched\\
|
|
\hline \hline
|
|
Pipeline & 40606 & 361 & 401 \\
|
|
\hline
|
|
Do & 46 & 26 & 401 \\
|
|
\hline
|
|
Await to Promise & 8 & 6 & 401 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with Atom source code}
|
|
\label{fig:evalAtom}
|
|
\end{figure} |