added results from evaluation
This commit is contained in:
parent
741cdcd658
commit
b74403fb2e
6 changed files with 145 additions and 4 deletions
BIN
build/report.pdf
BIN
build/report.pdf
Binary file not shown.
|
|
@ -330,7 +330,7 @@ When matching an expression the first statement in the program body of the AST g
|
|||
|
||||
In the case of the singular node in the body of the template program being a Statement, no removal has to be done, as a Statement can be used directly.
|
||||
|
||||
\subsubsection*{Recursively discovering matches}
|
||||
\paragraph{Recursively discovering matches}
|
||||
|
||||
The matcher used against single Expression/Statement templates is based upon a Depth-First Search in order to perform matching, and searches for matches from the top of the code definition. It is important we try to match against the template at all levels of the code AST, this is done by starting a new search one every child node of the code AST if the current node of the template tree is the top node of the template. This ensures we have tried to perform a match at any level of the tree, this also means we do not get any partial matches, as we only store matches that are returned at the recursive call when we do the search from the first node of the template tree.
|
||||
This is all done before ever checking the node we are currently on. The reason for this is to avoid missing matches that reside further down in the current branch, and also ensure matches further down are placed earlier in the full match array, which makes it easier to perform transformation when partial collisions exist.
|
||||
|
|
|
|||
|
|
@ -1 +1,96 @@
|
|||
\chapter{CH 5 goes here}
|
||||
\chapter{Evaluation}
|
||||
|
||||
In this chapter we will discuss how we evaluated \DSL and its related tools. This chapter will include some testing of the tool on demo code snippets, as well as running each of the proposals discussed in this thesis on some large scale JavaScript projects.
|
||||
|
||||
\section*{Testing on code}
|
||||
|
||||
In this section, we will showcase some synthetic transformations applied to code made to fit each of the definitions discussed in this thesis.
|
||||
|
||||
|
||||
The pipeline proposal is meant to merge the readability of chaining with the practicality of deeply nested calls. This deep nesting can be seen in the input code Listing \ref{ex:deepNestingEval}. The resulting code can be seen in Listing \ref{ex:transformedDeepNestingEval}.
|
||||
|
||||
\begin{lstlisting}[language={JavaScript}, label={ex:deepNestingEval}]
|
||||
// Original JavaScript
|
||||
a(a(a(a(a(a(a(b)))))));
|
||||
c(c(c(c(c(d, b), b), b), b), b);
|
||||
\end{lstlisting}
|
||||
\begin{lstlisting}[language={JavaScript}, label={ex:transformedDeepNestingEval}]
|
||||
// Transformed
|
||||
b |> a(%) |> a(%) |> a(%) |> a(%) |> a(%) |> a(%) |> a(%);
|
||||
d |> c(%, b) |> c(%, b) |> c(%, b) |> c(%, b) |> c(%, b);
|
||||
\end{lstlisting}
|
||||
|
||||
\section{Real Life source code}
|
||||
|
||||
In order to perform actual large scale trial of this program, we have collected some github projects containing many or large JavaScript files. Every JS file within the project is then passed through the entire tool, and we will evaluate it based upon the amount of matches discovered, as well as manual checking that the transformation resulted in correct code on the matches.
|
||||
|
||||
Each case study was evaluated by running this tool on every .js file in the repository, then collecting the number of matches found in total and how many files were successfully searched. Evaluating if the transformation was correct is done by manually sampling output files, and verifying that it passes through Babel Generate \cite{BabelGenerate} without error. You can see some highlighted transformations in Listing \textbf{INSERT FIGURE HERE}.
|
||||
|
||||
|
||||
"Pipeline"\cite{Pipeline} is clearly very applicable to most files, as it is really only looking for function calls. This is by far the best result, and it found matches in almost all files that Babel \cite{BabelParser} managed to parse.
|
||||
|
||||
The Do proposal \cite{Proposal:DoProposal} is expected to not find many matches, as code that has not been written in expression-oriented programming style will not produce many matches.
|
||||
|
||||
Await to promise also has an expected number of matches, but this evaluation proposal is not meant to be real life representative. As it is limited to functions containing only a single await statement and that statement has to be a \texttt{VariableDeclaration}.
|
||||
|
||||
\paragraph*{Next.js} is one of the largest projects related to JavaScript. It is owned by Vercel and is used as a development platform for the web.
|
||||
|
||||
\begin{figure}
|
||||
\begin{center}
|
||||
\begin{tabular}{|c|c|c|c|c|}
|
||||
\hline
|
||||
Proposal & Matches found & Files with matches & Files searched & Time\\
|
||||
\hline \hline
|
||||
Pipeline & 55787 & 1327 & 2331 & 89m 11s \\
|
||||
\hline
|
||||
Do & 18 & 15 & 2331 & 2m 12s \\
|
||||
\hline
|
||||
Await to Promise & 43 & 38 & 2331 & 16s \\
|
||||
\hline
|
||||
\end{tabular}
|
||||
\end{center}
|
||||
\caption{Evaluation with Next.js source code}
|
||||
\label{fig:evalNextJS}
|
||||
\end{figure}
|
||||
|
||||
\paragraph*{Three.js} is a library for 3D rendering in JavaScript. It is written purely in JavaScript and uses GPU for 3D calculations. It being a popular JavaScript library, and being written in mostly pure JavaScript makes it a good case study for our tool. It currently sits at over 1 million downloads weekly.
|
||||
|
||||
\begin{figure}
|
||||
\begin{center}
|
||||
\begin{tabular}{|c|c|c|c|c|}
|
||||
\hline
|
||||
Proposal & Matches found & Files with matches & Files searched & Time \\
|
||||
\hline \hline
|
||||
Pipeline & 85050 & 1119 & 1262 & 242m 58s \\
|
||||
\hline
|
||||
Do & 194 & 26 & 1262 & 6m 23s \\
|
||||
\hline
|
||||
Await to Promise & 125 & 80 & 1262 & 59s \\
|
||||
\hline
|
||||
\end{tabular}
|
||||
\end{center}
|
||||
\caption{Evaluation with Three.js source code}
|
||||
\label{fig:evalThreeJS}
|
||||
\end{figure}
|
||||
|
||||
\paragraph*{React} is a graphical user interface library for JavaScript, it facilitates the creation of user interfaces for both web and native platforms. React is based upon splitting a user interface into components for simple development. It is currently one of the most popular libraries for creating web apps and has over 223000 stars on Github.
|
||||
|
||||
\begin{figure}
|
||||
\begin{center}
|
||||
\begin{tabular}{|c|c|c|c|c|}
|
||||
\hline
|
||||
Proposal & Matches found & Files with matches & Files searched & Time\\
|
||||
\hline \hline
|
||||
Pipeline & 16353 & 1266 & 2051 & 84s \\
|
||||
\hline
|
||||
Do & 14 & 12 & 2051 & 8s \\
|
||||
\hline
|
||||
Await to Promise & 104 & 88 & 2051 & 6s \\
|
||||
\hline
|
||||
\end{tabular}
|
||||
\end{center}
|
||||
\caption{Evaluation with React source code}
|
||||
\label{fig:evalReact}
|
||||
\end{figure}
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -10,9 +10,9 @@ In AOP one creates an \textit{Aspect}, which is a module that contains some cros
|
|||
|
||||
Aspect oriented programming is similar to this project in that to define where \textit{Pointcuts} are placed, we have to define some structure and the AOP library has to search the code execution for events triggering the pointcut and run the advice defined within the aspect of that given pointcut. Essentially it performs a re-write of the code during execution to add functionality to multiple places in the executing code.
|
||||
|
||||
\section*{Other code-query languages}
|
||||
\section*{Other source code query languages}
|
||||
|
||||
In order to allow for simple analysis and refactoring of code, there already exists query languages for querying code. These languages use several methods to allow for querying code based on specific paradigms such as Logical queries, Declarative queries, or SQL like queries. All provide similar functionality of being able to query code. In this section we will look at three different languages for querying Java code, and their approach to searching code.
|
||||
In order to allow for simple analysis and refactoring of code, there already exists query languages for querying code. These languages use several methods to allow for querying code based on specific paradigms such as Logical queries, Declarative queries, or SQL like queries. All provide similar functionality of being able to query code. In this section we will look some of these languages for querying source code, and how they relate to \DSL developed in this thesis.
|
||||
|
||||
\subsection*{Browse-By-Query}
|
||||
|
||||
|
|
@ -28,10 +28,24 @@ Browse-By-Query is a language created for Java that analyses Java Bytecode files
|
|||
|
||||
PMD is the most versatile query language for Java source code querying out of all the ones explored in this section. \cite{ProgrammingLanguageEcolutionViaSourceCodeQueryLanguages}PMD supports querying of all Java constructs , it has this wide support due to constructing the entire codebase in XML format. This language was build for static code analysis, and therefore is a great way to perform queries on static code, it is mostly used as a tool for code editors to enforce programming styles.
|
||||
|
||||
\subsection*{Jackpot}
|
||||
|
||||
\cite{Jackpot}Jackpot is a query language created for the \cite{ApacheNetBeans}Apache Netbeans platform, it has since been mostly renamed to Java Declarative Hints Language, we will continue to refer to it as Jackpot in this section. The language uses declarative patterns to define source code queries, these queries are used in conjunction with multiple rewrite definitions. This is used in the Apache Netbeans suite of tools to allow for declarative refactoring of code.
|
||||
|
||||
This is quite similar to the form of \DSL, as both language define som query by using similar structure, in Jackpot you define a \textit{pattern}, then every match of that pattern can be re-written to a \textit{fix-pattern}, each fix-pattern can have a condition attached to it. This is quite similar to the \textit{applicable to} and \textit{transform to} sections of \DSL. Jackpot also supports something similar to the wildcards in \DSL, as you can define variables in the \textit{pattern} definition and transfer them over to the \textit{fix-pattern} definition. This is closely related to the definition of wildcards in \DSL, though without type restrictions and notation for matching more than one AST node.
|
||||
|
||||
|
||||
|
||||
\section*{JetBrains structural search}
|
||||
|
||||
JetBrains integrated development environments have a feature that allows for \cite{StructuralSearchAndReplaceJetbrains} structural search and replace. This feature is intended for large code bases where a developer wants to perform a search and replace based on syntax and semantics, not just a regular text based search and replace. A search is applied to specific files of the codebase or the entire codebase. It does not recursively check the entire static structure of the code, but this can be specified in the user interface of structural search and replace.
|
||||
|
||||
When doing structural search in Jetbrains IntelliJ IDEA, templates are used to describe the query used in the search. These templates use variables described with \texttt{\$variable\$}, these allow for transferring context to the structural replace.
|
||||
|
||||
This tool is an interactive exprience, where each match is showcased in the find tool, and the developer can decide which matches to apply the replace template to. This allows for error avoidance and a stricter search that is verified by humans. If the developer wants, they do not have to verify each match and just replace everything.
|
||||
|
||||
When comparing this tool to \DSL and its corresponding program, there are some similarities. They are both template based, which means a search uses a template to define query, both templates contain variables/wildcards in order to match against a free section, and the replacing structure is also a template based upon those same variables. A way of matching the variables/wildcards of structural search and replace also exists, one can define the amount of X node to match against, similar to the \texttt{+} operator used in \DSL. A core difference between \DSL and structural search and replace is the variable type system. When performing a match and transformation in \DSL the types are used extensively to limit the match against the wildcards, while this limitation is not possible in structural search and replace.
|
||||
|
||||
|
||||
\section{Other JavaScript parsers}
|
||||
|
||||
|
|
@ -43,6 +57,8 @@ Similar to \cite{Babel}Babel, Speedy Web Compiler is an extensible parser that a
|
|||
|
||||
Speedy Web Compiler supports features out of the box such as Compilation, used for TypeScript and other languages that are compiled down to JavaScript. Bundling, which takes multiple JavaScript/TypeScript files and bundles them into a single output file, while handling naming collisions. Minification, to make the bundle size of a project smaller, transforming for use with WebAssembly, and custom plugins to change the specification of the languages parsed by SWC.
|
||||
|
||||
Compared to Babel used in this paper, SWC focuses on speed, as its main selling point is a faster way of developing web projects.
|
||||
|
||||
\subsection*{Acorn}
|
||||
|
||||
Acorn is another parser written in JavaScript to parser JavaScript and it's related languages. Acorn focuses on plugin support in order to support extending and redefinition on how it's internal parser works. It has a very similar syntax to and has it's own tree traversal library Acorn Walk. \cite{Babel}Babel is originally a fork of Acorn, while Babel has since had a full rewrite. Acorn focuses heavily on supporting third party plugins, which Babel does not. However Acorn was not a good fit for this project, as Acorn only supports Stage 4 proposals, and support for proposals in the early stages is a requirement.
|
||||
|
|
@ -216,3 +216,32 @@
|
|||
location = {Tucson, Arizona, USA},
|
||||
series = {PLATEAU '12}
|
||||
}
|
||||
@misc{Jackpot,
|
||||
author = {NetBeans, Apache},
|
||||
title = {{Java Declarative Hints Language}},
|
||||
year = {2024},
|
||||
month = mar,
|
||||
urldate = {2024-05-21},
|
||||
note = {[Online; accessed 21. May 2024]},
|
||||
url = {https://netbeans.apache.org/front/main/jackpot/HintsFileFormat/#variables}
|
||||
}
|
||||
|
||||
@misc{ApacheNetBeans,
|
||||
author = {NetBeans, Apache},
|
||||
title = {{Welcome to Apache NetBeans}},
|
||||
year = {2024},
|
||||
month = feb,
|
||||
urldate = {2024-05-21},
|
||||
note = {[Online; accessed 21. May 2024]},
|
||||
url = {https://netbeans.apache.org/front/main/index.html}
|
||||
}
|
||||
|
||||
@misc{StructuralSearchAndReplaceJetbrains,
|
||||
title = {{Structural search and replace {$\vert$} IntelliJ IDEA}},
|
||||
journal = {IntelliJ IDEA Help},
|
||||
year = {2024},
|
||||
month = apr,
|
||||
urldate = {2024-05-22},
|
||||
note = {[Online; accessed 22. May 2024]},
|
||||
url = {https://www.jetbrains.com/help/idea/structural-search-and-replace.html}
|
||||
}
|
||||
|
|
@ -18,6 +18,7 @@
|
|||
\include{chapter/ch2}
|
||||
\include{chapter/ch3}
|
||||
\include{chapter/ch4}
|
||||
\include{chapter/ch5}
|
||||
\include{chapter/related_work}
|
||||
\include{chapter/future_work}
|
||||
% Include more chapters as required.
|
||||
|
|
|
|||
Loading…
Reference in a new issue