238 lines
9.4 KiB
TeX
238 lines
9.4 KiB
TeX
\chapter{Evaluation}
|
|
|
|
In this chapter we will discuss how we evaluated \DSL and its related tools. This chapter will include some testing of the tool on demo code snippets, as well as running each of the proposals discussed in this thesis on some large scale JavaScript projects.
|
|
|
|
\section{Real Life source code}
|
|
|
|
To perform actual large scale trial of this program, we have collected some github projects containing many or large JavaScript files. Every JS file within the project is then passed through the entire tool, and we will evaluate it based upon the amount of matches discovered, as well as manual checking that the transformation resulted in correct code on the matches.
|
|
|
|
Each case study was evaluated by running this tool on every .js file in the repository, then collecting the number of matches found in total and how many files were successfully searched. Evaluating if the transformation was correct is done by manually sampling output files, and verifying that it passes through Babel Generate~\cite{BabelGenerate} without error.
|
|
|
|
"Pipeline"~\cite{Pipeline} is very applicable to most files, as the concept it touches (fucntion calls) is widely used all across JavaScript. This is by far the best result, and it found matches in almost all files that Babel~\cite{BabelParser} managed to parse.
|
|
|
|
The Do proposal~\cite{Proposal:DoProposal} is expected to not find as many matches, as code that has not been written in expression-oriented programming style will not produce many matches. However, this also highlights how impactful this proposal is to previously written code compared to "Pipeline".
|
|
|
|
Await to promise also has an expected number of matches, but this evaluation proposal is not meant to be real life representative. As it is limited to functions containing only a single await statement and that statement has to be a \texttt{VariableDeclaration}.
|
|
|
|
All these proposals have an impact on JavaScript, as they change how certain functionality is achieved. However, the size of the impact is very different. "Pipeline" is applicable to all function calls, this is why it produces so many more transformations than the others. From this we can deduce the impact each proposal might have on the current version of JavaScript. "Do Expression" is an interesting case, as it does produce a number of matches, and is quite applicable when ran on a large codebase. ... More
|
|
|
|
\paragraph*{Next.js}~\cite{NEXT.JS} is one of the largest projects on the web. It is used with React~\cite{React} to enable feature such as server-sire rendering and static site generation.
|
|
|
|
\begin{table}[H]
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files processed\\
|
|
\hline \hline
|
|
"Pipeline" & 242079 & 1912 & 3340 \\
|
|
\hline
|
|
"Do" expression & 480 & 111 & 3340 \\
|
|
\hline
|
|
Await to Promise & 143 & 75 & 3340 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with Next.js source code}
|
|
\label{fig:evalNextJS}
|
|
\end{table}
|
|
|
|
\begin{figure}[H]
|
|
\begin{lstlisting}[language={JavaScript}]
|
|
async function getCurrentRules() {
|
|
return fetch(`https://api.github.com/repos/vercel/next.js/branches/canary/protection`, {
|
|
headers: {
|
|
Accept: 'application/vnd.github+json',
|
|
Authorization: `Bearer ${authToken}`,
|
|
'X-GitHub-Api-Version': '2022-11-28'
|
|
}
|
|
}).then(async res => {
|
|
if (!res.ok) {
|
|
throw new Error(`Failed to check for rule ${res.status} ${await res.text()}`);
|
|
}
|
|
const data = await res.json();
|
|
return {
|
|
// Massive JS object
|
|
};
|
|
});
|
|
}
|
|
\end{lstlisting}
|
|
\caption*{"Await to Promise" transformation, from \texttt{next.js/test/integration/typescript-hmr/index.test.js}}
|
|
\end{figure}
|
|
|
|
\begin{figure}[H]
|
|
\begin{lstlisting}[language={JavaScript}]
|
|
for (const file of typeFiles) {
|
|
const content = await fs.readFile(join(styledJsxPath, file), 'utf8')
|
|
await fs.writeFile(join(typesDir, file), content)
|
|
}
|
|
\end{lstlisting}
|
|
\begin{lstlisting}[language={JavaScript}]
|
|
for (const file of typeFiles) {
|
|
const content = await (styledJsxPath |> join(%, file) |> fs.readFile(%, 'utf8'));
|
|
await (typesDir |> join(%, file) |> fs.writeFile(%, content));
|
|
}
|
|
\end{lstlisting}
|
|
\caption*{"Pipeline" transformation, taken from \texttt{next.js/packages/next/taskfile.js}}
|
|
\end{figure}
|
|
|
|
\begin{figure}[H]
|
|
\begin{lstlisting}[language={JavaScript}]
|
|
await check(async () => {
|
|
const html = await browser.eval('document.documentElement.innerHTML')
|
|
return html.match(/iframe/) ? 'fail' : 'success'
|
|
}, /success/)
|
|
\end{lstlisting}
|
|
\begin{lstlisting}[language={JavaScript}]
|
|
await check(do {
|
|
const html = await browser.eval('document.documentElement.innerHTML');
|
|
html.match(/iframe/) ? 'fail' : 'success'
|
|
}, /success/);
|
|
\end{lstlisting}
|
|
\caption*{"Do expression" transformation, taken from \texttt{next.js/test/integration/typescript-hmr/index.test.js}}
|
|
\end{figure}
|
|
|
|
|
|
|
|
\paragraph*{Three.js}~\cite{ThreeJS} is a library for 3D rendering in JavaScript. It is written purely in JavaScript and uses GPU for 3D calculations. It being a popular JavaScript library, and being written in mostly pure JavaScript makes it a good case study for our tool. It currently sits at over 1 million downloads weekly.
|
|
|
|
\begin{table}[H]
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched \\
|
|
\hline \hline
|
|
Pipeline & 84803 & 1117 & 1384 \\
|
|
\hline
|
|
"Do" expression & 277 & 55 & 1384 \\
|
|
\hline
|
|
Await to Promise & 13 & 7 & 1384 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with Three.js source code}
|
|
\label{fig:evalThreeJS}
|
|
\end{table}
|
|
|
|
\begin{figure}[H]
|
|
\begin{lstlisting}[language={JavaScript}]
|
|
tracks.push( parseKeyframeTrack( jsonTracks[ i ] ).scale( frameTime ) );
|
|
\end{lstlisting}
|
|
\begin{lstlisting}[language={JavaScript}]
|
|
frameTime
|
|
|> (jsonTracks[i] |> parseKeyframeTrack(%)).scale(%)
|
|
|> tracks.push(%);
|
|
\end{lstlisting}
|
|
\caption*{Transformation taken from \texttt{three.js/src/animation/AnimationClip.js}}
|
|
\end{figure}
|
|
|
|
\paragraph*{React}~\cite{React} is a graphical user interface library for JavaScript, it facilitates the creation of user interfaces for both web and native platforms. React is based upon splitting a user interface into components for simple development. It is currently one of the most popular libraries for creating web apps and has over 223000 stars on Github.
|
|
|
|
\begin{table}[H]
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched\\
|
|
\hline \hline
|
|
"Pipeline" & 16353 & 1266 & 2051 \\
|
|
\hline
|
|
"Do" expression & 79 & 60 & 2051 \\
|
|
\hline
|
|
Await to Promise & 30 & 13 & 2051 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with React source code}
|
|
\label{fig:evalReact}
|
|
\end{table}
|
|
|
|
\begin{figure}[H]
|
|
\begin{lstlisting}[language={JavaScript}]
|
|
const logger = createLogger({
|
|
storagePath: join(__dirname, '.progress-estimator'),
|
|
});
|
|
\end{lstlisting}
|
|
\begin{lstlisting}[language={JavaScript}]
|
|
const logger = {
|
|
storagePath: __dirname |> join(%, '.progress-estimator')
|
|
} |> createLogger(%);
|
|
\end{lstlisting}
|
|
\caption*{"Pipeline" transformation, taken from \texttt{react/scripts/devtools/utils.js}}
|
|
\end{figure}
|
|
|
|
|
|
|
|
\paragraph*{Bootstrap}~\cite{Bootstrap} is a front-end framework used for creating responsive and mobile-first websites, it comes with a variety of built-in components, as well as a built in styling. This styling is also customizable using CSS. This library is a good evaluation point for this thesis as it is written in pure JavaScript and is used by millions of developers.
|
|
|
|
\begin{table}[H]
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched\\
|
|
\hline \hline
|
|
""Pipeline" & 13794 & 109 & 115 \\
|
|
\hline
|
|
"Do" expression & 13 & 8 & 115 \\
|
|
\hline
|
|
Await to Promise & 0 & 0 & 115 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with Bootstrap source code}
|
|
\label{fig:evalBootstrap}
|
|
\end{table}
|
|
|
|
\begin{figure}[H]
|
|
\begin{lstlisting}[language={JavaScript}]
|
|
if (isElement(content)) {
|
|
this._putElementInTemplate(getElement(content), templateElement)
|
|
return
|
|
}
|
|
\end{lstlisting}
|
|
\begin{lstlisting}[language={JavaScript}]
|
|
if (content |> isElement(%)) {
|
|
content |> getElement(%) |> this._putElementInTemplate(%, templateElement);
|
|
return;
|
|
}
|
|
\end{lstlisting}
|
|
\caption*{"Pipeline" transformation, taken from \texttt{bootstrap/js/src/util/template-factory.js}}
|
|
\end{figure}
|
|
|
|
|
|
|
|
\paragraph*{Atom}~\cite{Atom} is a text editor made in JavaScript using the Electron framework. It was created to give a very minimal and modular text editor. It was bought by Microsoft, and later discontinued in favor for Visual Studio Code.
|
|
|
|
\begin{table}[H]
|
|
\begin{center}
|
|
\begin{tabular}{|c|c|c|c|}
|
|
\hline
|
|
Proposal & Matches found & Files with matches & Files searched\\
|
|
\hline \hline
|
|
"Pipeline" & 40606 & 361 & 401 \\
|
|
\hline
|
|
"Do" expression & 46 & 26 & 401 \\
|
|
\hline
|
|
Await to Promise & 12 & 7 & 401 \\
|
|
\hline
|
|
\end{tabular}
|
|
\end{center}
|
|
\caption{Evaluation with Atom source code}
|
|
\label{fig:evalAtom}
|
|
\end{table}
|
|
|
|
|
|
\begin{figure}[H]
|
|
\begin{lstlisting}[language={JavaScript}]
|
|
if (repo && repo.onDidDestroy) {
|
|
repo.onDidDestroy(() =>
|
|
this.repositoryPromisesByPath.delete(pathForDirectory)
|
|
);
|
|
}
|
|
\end{lstlisting}
|
|
\begin{lstlisting}[language={JavaScript}]
|
|
if (repo && repo.onDidDestroy) {
|
|
(() => pathForDirectory |> this.repositoryPromisesByPath.delete(%)) |> repo.onDidDestroy(%);
|
|
}
|
|
\end{lstlisting}
|
|
\caption*{"Pipeline" transformation, taken from \texttt{atom/src/project.js}}
|
|
\end{figure}
|
|
|