Almost finished re-writing chapter 3 feedback

This commit is contained in:
Rolf Martin Glomsrud 2024-05-25 16:23:42 +02:00
parent 1d5fa0efc1
commit 02dd10ce48
3 changed files with 159 additions and 125 deletions

Binary file not shown.

View file

@ -290,9 +290,7 @@ The Do Expressions\cite{Proposal:DoProposal} proposal, is a proposal meant to br
The motivation of the "Do expression" proposal is to allow for local scoping of a code block that is treated as an expression. Thus, complex code requiring multiple statements will be confined inside its own scope\cite{ScopeECMA-262} and the resulting value is returned from the block implicitly as an expression, similarly to how a unnamed functions or arrow functions are currently used. In order to achieve this behavior in the current stable version of ECMAScript, one needs to use unnamed functions and invoke them immediately, or use an arrow function.
The codeblock of a \texttt{do} expression has one major difference from these equivalent functions, as it allows for implicit return of the final expression of the block, and is the resulting value of the entire \texttt{do} expression. This gives that the final statement of a \texttt{do} expression has to contain a completion value \cite{CompletionRecord}, where the value will be the result of the expression, and cannot be an \texttt{if} without an \texttt{else}, a loop, or a variable declaration.
The local scoping of this feature allows for a cleaner environment in the parent scope of the \texttt{do} expression. What is meant by this is for temporary variables and other assignments used once can be enclosed inside a limited scope within the \texttt{do} block. Allowing for a cleaner environment inside the parent scope where the \texttt{do} block is defined.
The codeblock of a \texttt{do} expression has one major difference from these equivalent functions, as it allows for implicit return of the final statement of the block, and is the resulting value of the entire \texttt{do} expression. The local scoping of this feature allows for a cleaner environment in the parent scope of the \texttt{do} expression. What is meant by this is for temporary variables and other assignments used once can be enclosed inside a limited scope within the \texttt{do} block. Allowing for a cleaner environment inside the parent scope where the \texttt{do} block is defined.
\noindent\begin{minipage}{.45\textwidth}
\begin{lstlisting}[language={JavaScript}]
@ -335,15 +333,12 @@ let x = do {
\end{minipage}\hfil
This proposal has some limitations on its usage. Due to the implicit return of the final expression you cannot end a do expression with an \texttt{if} without and \texttt{else}, or a \texttt{loop}.
\subsection{Await to Promise}
This section covers an imaginary proposal that was used to evaluate the program developed in this thesis. This imaginary proposal is less of a proposal and more of just a pure JavaScript transformation example. What this proposal wants to achieve is transforming a function using \texttt{await}, into a function that uses and returns a promise.
We discuss now an imaginary proposal that was used as a running example during the development of this thesis. This proposal is of just a pure JavaScript transformation example. The transformation this proposal is meant to display, is transforming a code using \texttt{await}\cite{Await}, into code which uses a promise\cite{Promise}.
In order to do this an equivalent way of writing code containing \texttt{await} in the syntax of \texttt{.then()} promise, an equivalent way of writing the same functionality has to be identified. In this case, the equivalent way of expressing this using a promise is consuming the rest of the scope after \texttt{await} was written, and place it inside a \texttt{then(() => {})} function. The variable the await was assigned to has to be used as the argument to the \texttt{.then()} function.
To perform this transformation, we define an equivalent way of expressing an \texttt{await} expression as a promise. The equivalent way of expressing \texttt{await} with a promise, is removing \texttt{await} from the expression, this expression now will return a promise, which has a function \texttt{then()}, this function is executed when the promise resolves. We pass an arrow function as argument to \texttt{then()}, and append each following statement in the current scope\cite{ScopeECMA-262} inside the block of that arrow function. This will result in equivalent behavior to using \texttt{await}.
\noindent\begin{minipage}{.45\textwidth}
\begin{lstlisting}[language={JavaScript}]
@ -372,22 +367,91 @@ async function a(){
\section{Searching user code for applicable snippets}
In order to identify snippets of code in the users codebase where a proposal is applicable we need some way to define patterns of code where we can apply the proposal. To do this, a DSL titled \DSL is used.
In order to identify snippets of code in the user's code where a proposal is applicable, we need some way to define patterns of code to use as a query. To do this, we have designed and implemented a domain-specific language that allows matching parts of code that is applicable to some proposal, and transforming those parts to use the features of that proposal.
\subsection{Structure of \DSL}
\label{sec:DSLStructure}
\paragraph*{Proposal Definition}
\DSL is designed to mimic the examples already provided in proposal descriptions \cite{TC39Process}. These examples can be seen in each of the proposals described in Section \ref{sec:proposals}. The idea is to allow a similar kind of notation to the examples in order to define the transformations.
The first part of \DSL is defining the proposal, this is done by creating a named block containing all definitions of templates used for matching alongside their respective transformation. This section is used to contain everything relating to a specific proposal and is meant for easy proposal identification by tooling.
\begin{lstlisting}
proposal Pipeline_Proposal {}
\end{lstlisting}
\paragraph*{Case}
Each proposal will have one or more definitions of a template for code to identify in the users codebase, and its corresponding transformation definition. These are grouped together in order to have a simple way of identifying the corresponding cases of matching and transformations. This section of the proposal is defined by the keyword \textit{case} and a block that contains its related fields. A proposal definition in \DSL should contain at least one \texttt{case} definition. This allows for matching many different code snippets and showcasing more of the proposal than a single concept the proposal has to offer.
\begin{lstlisting}
case case_name {
}
\end{lstlisting}
\paragraph*{Template used for matching}
In order to define the template used to match, we have another section defined by the keyword \textit{applicable to}. This section will contain the template defined using JavaScript with specific DSL keywords defined inside the template. This template is used to identify applicable parts of the user's code to a proposal.
\begin{lstlisting}
applicable to {
"let a = 0;"
}
\end{lstlisting}
\paragraph*{Defining the transformation}
In order to define the transformation that is applied to a specific matched code snippet, the keyword \textit{transform to} is used. This section is similar to the template section, however it uses the specific DSL identifiers defined in applicable to, in order to transfer the context of the matched user code, this allows us to keep parts of the users code important to the original context it was written in.
\begin{lstlisting}
transform to{
"() => {
let b = 100;
}"
}
\end{lstlisting}
\paragraph*{Full definition of \DSL}
Taking all these parts of \DSL structure, defining a proposal in \DSL will look as follows.
\begin{lstlisting}[caption={\DSL definition of a proposal}]
proposal PROPOSAL_NAME {
case CASE_NAME_1 {
applicable to {
"let b = 100;"
}
transform to {
"() => {};"
}
}
case CASE_NAME_2 {
applicable to {
"console.log();"
}
transform to {
"console.dir();"
}
}
}
\end{lstlisting}
\subsection{\DSL}
\label{sec:DSL_DEF}
Showcasing a proposal using a users code requires some way of identifying applicable code sections to that proposal. To do this, we have designed a DSL called \DSL , JavaScript Template Query Language. This DSL will contain the entire definition used to identify and transform user code in order to showcase a proposal.
Showcasing a proposal using a user's code requires some way of identifying applicable code sections to that proposal. To do this, we have designed a DSL called \DSL, JavaScript Template Query Language.
\subsection*{Identifying applicable code}
In order to identify sections of code a proposal is applicable to, we use templates of JavaScript. These templates are used to identify and match applicable sections of a users code. A matching section for a template is one that produces an exactly equal AST structure, where each node of the AST sections has the same information contained within it. This means templates are matched exactly against the users code, this does not really provide some way of actually querying the code and performing context based transformations, so for that we use \textit{Wildcards} within the template.
In order to identify sections of code a proposal is applicable to, we use \emph{templates}, which are snippets of JavaScript. These templates are used to identify and match applicable sections of a users code. A matching section for a template is one that produces an exactly equal AST structure, where each node of the AST sections has the same information contained within it. This means that templates are matched exactly against the users code, this does not really provide some way of querying the code and performing context based transformations, so for that we use \textit{wildcards} within the template.
\textit{Wildcards} are written into the template inside a block denoted by $<$$<$ $>$$>$. Each wildcard has to start with an identifier, which is a way of referring to that wildcard in the definition of the transformation template later. This allows for transferring the context of parts matched to a wildcard into the transformed output, like identifiers, parts of statements or even entire statements all together can be transferred from the original user code into the transformation template.The second part of the wildcard contains a wildcard type expression. A wildcard type expression is a way of defining exactly what types of AST nodes a wildcard will produce a match against, these type expressions use boolean logic together with the AST node-types from \cite{Babel}{BabelJS} to create a very strict way of defining wildcards.
Wildcards are interspliced into the template inside a block denoted by \texttt{$<$$<$ $>$$>$}. Each wildcard starts with an identifier, which is a way of referring to that wildcard in the definition of the transformation template later. This allows for transferring the context of parts matched to a wildcard into the transformed output, like identifiers, parts of statements, or even entire statements, can be transferred from the original user code into the transformation template. A wildcard also contains a type expression. A type expression is a way of defining exactly the types of AST nodes a wildcard will produce a match against. These type expressions use Boolean logic together with the AST node-types from BabelJS\cite{Babel} to create a very versatile of defining exactly what nodes a wildcard can match against.
\subsubsection*{Wildcard type expressions}
Wildcard type expressions allow for writing complex boolean logic on the kinds of nodes a wildcard can be matched against, this means writing the type for a wildcard can be as simple as just \texttt{Expression || Statement}, or as complex as \texttt{((Statement \&\& ! ReturnStatement) \&\& !VariableDeclaration)*}. The operators mean the following, \texttt{\&\&} is logical AND, this means both parts of the expression have to evaluate to true, \texttt{||} means logical OR, so either side of expression can be true for the entire expression to be true, \texttt{!} is the only unary expression, and is logical NOT, so \texttt{!Statement} is any node that is NOT a Statement. There is also this definition \texttt{+}, this is only valid at the top level of the expression, and the pluss operator means this wildcard can be used any number of times in order. This is useful for matching against a series of Statements, while not wanting to match an entire BlockStatement. The final part of a wildcard expression is the AST types used, an example of this would be the wildcard type expression \texttt{ReturnStatement}, which will only match against an AST node of type ReturnStatement. Using the power of the wildcards, we can create templates that can be used for querying a users code for specific code sections that a proposal is applicable to.
Wildcard expressions are used to match AST node types based on Boolean logic. This means an expression can be as simple as \texttt{VariableDeclaration}: this will match only against a node of type \texttt{VariableDeclaration}. Every type used in these expressions are compared against the AST node types from Babel\cite{Babel}, meaning every AST node type is supported. We also include the types \texttt{Statement} for matching against a statement, and \texttt{Expression} for matching any expression. The expressions also support binary and unary operators, an example \texttt{Statement \&\& !ReturnStatement} will match any statement which is not of type \texttt{ReturnStatement}. The expressions support the following operators, \texttt{\&\&} is logical AND, this means both parts of the expression have to evaluate to true, \texttt{||} means logical OR, so either side of expression can be true for the entire expression to be true, \texttt{!} is the only unary expression, and is logical NOT, so \texttt{!Statement} is any node that is NOT a Statement. The wildcards support matching multiple sibling nodes, this is done by using \texttt{(expr)+}, this is only valid at the top level of the expression. This is useful for matching against a series of one or more Statements, while not wanting to match an entire \texttt{BlockStatement}, this is written as \texttt{(Statement \&\& !ReturnStatement)+}.
\begin{lstlisting}
@ -399,129 +463,38 @@ A wildcard section is defined on the right hand side of an assignment statement.
\subsection{Transforming}
When matching sections of the users code has been found, we need some way of defining how to transform those sections to showcase a proposal. This is done by a similar template to applicable to, namely \textit{transform to}, this template describes the general structure of the newly transformed code.
When matching sections of the users code has been found, we need some way of defining how to transform those sections to showcase a proposal. This is done in an \texttt{transform to} block, this template describes the general structure of the newly transformed code.
A transformation template is used to define how the matches will be transformed after applicable code has been found. The transformation is a general template of the code once the match is replaced in the original AST. However, without transferring over the context from the match, it is just a template search and replace. So in order to transfer the context from the match, wildcards are defined in this template as well. These wildcards use the same block notation found in the applicable to template, however they do not need to contain the types, as those are not needed in the transformation. The only section required in the wildcard is the identifier used in applicable to, this is done in order to know which wildcard match we are taking the context from, and where to place it in the transformation template.
A transformation template is used to define how the matches will be transformed after applicable code has been found. The transformation is a general template of the code once the match is replaced in the original AST. However, without transferring over the context from the match, this would be a template search and replace. Thus, in order to transfer the context from the match, wildcards are defined in this template as well. These wildcards use the same block notation found in the \texttt{applicable to} template, however they do not need to contain the types, as those are not needed in the transformation. The only required field of the wildcard is the identifier defined in \texttt{applicable to}. This is done in order to know which wildcard match we are taking the context from, and where to place it in the transformation template.
Transforming a variable declaration from using \texttt{let} to use \texttt{const}.
\begin{lstlisting}[language={JavaScript}]
// Example of transform to template
const variableName = <<expr1>>;
\end{lstlisting}
\subsection{Structure of \DSL}
\label{sec:DSLStructure}
\DSL is designed to mimic the examples already provided by a proposal champion in the proposals README. These examples can be seen in each of the proposals described in \ref{sec:proposals}. The idea is to allow a similar kind of notation to the examples in order to define the transformations.
\subsubsection*{Define proposal}
The first part of \DSL is defining the proposal, this is done by creating a named block containing all definitions of templates used for matching alongside their respective transformation. This section is used to contain everything relating to a specific proposal and is meant for easy proposal identification by tooling.
\begin{lstlisting}[caption={Example of section containing the pipeline proposal}]
proposal Pipeline_Proposal{
}
\end{lstlisting}
\subsubsection*{Defining a pair of template and transformation}
Each proposal will have 1 or more definitions of a template for code to identify in the users codebase, and its corresponding transformation definition. These are grouped together in order to have a simple way of identifying the corresponding cases of matching and transformations. This section of the proposal is defined by the keyword \textit{case} and a block to contain its related fields. A proposal will contain 1 or more of this section. This allows for matching many different code snippets and showcasing more of the proposal than a single concept the proposal has to offer.
\begin{lstlisting}[caption={Example of pair section}]
case case_name {
}
\end{lstlisting}
\subsubsection*{Template used for matching}
In order to define the template used to match, we have another section defined by the keyword \textit{applicable to}. This section will contain the template defined using JavaScript with specific DSL keywords defined inside the template.
\begin{lstlisting}[caption={Example of applicable to section}]
// Example applicable to template
applicable to {
let <<variableName: Identifier>> = <<expr1: Expression>>;
}
// Example of transform to template
transform to {
const <<variableName>> = <<expr1>>;
}
\end{lstlisting}
\subsubsection*{Defining the transformation}
In order to define the transformation that is applied to a specific matched code snippet, the keyword \textit{transform to} is used. This section is similar to the template section, however it uses the specific DSL identifiers defined in applicable to, in order to transfer the context of the matched user code, this allows us to keep parts of the users code important to the original context it was written in.
\begin{lstlisting}[caption={Example of transform to section}]
transform to{
}
\end{lstlisting}
\subsubsection*{All sections together}
Taking all these parts of \DSL structure, defining a proposal in \DSL will look as follows.
\begin{lstlisting}[caption={\DSL definition of a proposal}]
proposal PROPOSAL_NAME {
case PAIR_NAME {
applicable to {
}
transform to {
}
}
pair PAIR_NAME {
applicable to .....
}
pair ....
}
\end{lstlisting}
\section{\DSLSH}
In this thesis, we also created an alternative way of defining proposals and their respective transformations, this is done using JavaScript as it's own meta language for the definitions. The reason for creating a way of defining proposals using JavaScript is, it allows us to limit the amount of dependencies of the tool, since we no longer rely on \DSL, and it allows for more exploration in the future work of this project.
\DSLSH is less of an actual language, and more of a program API at the moment, it allows for defining proposals purely in JavaScript objects, which is meant to allow a more modular way of using this idea. In \DSLSH you define a \textit{prelude}, which is just a list of variable declarations that contain the type expression as a string for that given wildcard. This means we do not need to perform wildcard extraction when wanting to parse the templates used for matching and transformation.
\noindent\begin{minipage}{.45\textwidth}
\begin{lstlisting}
// Definition in JSTQL
proposal a{
case {
applicable to {
<<a:Expression>>
}
transform to {
() => <<a>>
}
}
}
\end{lstlisting}
\end{minipage}\hfil
\noindent\begin{minipage}{.45\textwidth}
\begin{lstlisting}[language={JavaScript}]
// Equivalent definition in JSTQL-SH
{
prelude: 'let a = "Expression"'`,
applicableTo: "a;",
transformTo: "() => a;"
}
\end{lstlisting}
\end{minipage}\hfil
\section{Using the \DSL with an actual syntactic proposal}
\section{Using the \DSL with syntactic proposals}
This section contains the definitions of the proposals used to evaluate the tool created in this thesis. These definitions do not have to cover every single case where the proposal might be applicable, as they just have to be general enough to create some amount of examples that will give a representative number of matches when the transformations are applied to some relatively long user code.
\subsection{Pipeline Proposal}
\subsection{"Pipeline" Proposal}
The Pipeline Proposal is the easiest to define of the proposals presented in \ref*{sec:proposals}. This is due to the proposal being applicable to a very wide array of expressions, and the main problem this proposal is trying to solve is deep nesting of function calls.
The Pipeline proposal is the first we define of the proposals presented in Section \ref*{sec:proposals}. This is due to the proposal being applicable to function calls, which is used all across JavaScript. This proposal is trying to solve readability when performing deeply nested function calls.
\begin{lstlisting}[language={JavaScript}, caption={Example of Pipeline Proposal definition in \DSL}, label={def:pipeline}]
proposal Pipeline{
proposal Pipeline {
case SingleArgument {
applicable to {
@ -533,7 +506,7 @@ proposal Pipeline{
}
}
case DualArgument{
case TwoArgument{
applicable to {
"<<someFunctionIdent: Identifier || MemberExpression>>(<<someFunctionParam: Expression>>, <<moreFunctionParam: Expression>>)"
}
@ -544,11 +517,12 @@ proposal Pipeline{
}
\end{lstlisting}
This first pair definition \texttt{SingleArgument} of the Pipeline proposal will apply to any \textit{CallExpression} with a single argument. And it will be applied to each of the deeply nested callExpressions in a nested call. The second pair definition \texttt{DualArgument} will apply to any \textit{CallExpression} with 2 arguments. One can in theory define any number of cases to cover a higher amount of arguments in the function call, however we only need to cover enough cases to produce at least some matches, so a smaller definition is better in this case
In the Listing \ref*{def:pipeline}, the first pair definition \texttt{SingleArgument} will apply to any \textit{CallExpression} with a single argument. We do not expressively write a \texttt{CallExpression} inside a wildcard, as we have defined the structure of a \texttt{CallExpression}. The first wildcard \texttt{someFunctionIdent}, has the types of \texttt{Identifier}, to match against single identifiers, and \texttt{MemberExpression}, to match against functions who are members of objects, i.e. \texttt{console.log}. In the transformation template, we define the structure of a function call using the pipe operator, but the wildcards change order, so the argument passed as argument \texttt{someFunctionParam} is placed on the left side of the pipe operator, and the \texttt{CallExpression} is on the right, with the topic token as the argument. This case will produce a match against all function calls with a single argument, and transform them to use the pipe operator. The main difference of the second case \texttt{TwoArgument}, is it matches against functions with exactly two arguments, and uses the first argument as the left side of the pipe operator, while the second argument remains in the function call.
\subsection{Do Proposal}
The \cite[Do Proposal]{Proposal:DoProposal} can also be defined with this tool. This definition will never catch all the applicable sections of the users code, and is very limited in where it might discover this proposal is applicable. This is due to the Do Proposal introducing an entirely new way to write JavaScript (Expression-oriented programming). If the user running this tool has not used the current status-quo way of doing expression-oriented programming in JavaScript, \DSL will probably not find any applicable snippets in the users code. However, in a reasonably large codebase, some examples will probably be discovered.
\subsection{"Do Expressions" Proposal}
The "Do Expressions" proposal\cite{Proposal:DoProposal} can be specified in our tool. Due to the nature of the proposal, it is not as applicable as the "Pipeline" proposal, as it does not re-define a style that is used quite as frequently as call expressions. This means the amount of transformed code snippets this specification in \DSL will be able to perform is expected to be lower. This is due to the Do Proposal introducing an entirely new way to write expression-oriented code in JavaScript. If the user running this tool has not used the current way of writing in an expression-oriented style in JavaScript, \DSL is limited in the amount of transformations it can perform. Nevertheless, if the user has been using an expression-oriented style, \DSL will transform parts of the code.
\begin{lstlisting}[language={JavaScript}, caption={Definition of Do Proposal in \DSL}, label={def:doExpression}]
proposal DoExpression{
@ -587,6 +561,7 @@ proposal DoExpression{
\end{lstlisting}
\subsection{Await to Promises evaluation proposal}
This section will cover the evaluation proposal we created in order to evaluate this tool described in \ref{sec:proposals}.
@ -611,4 +586,36 @@ proposal awaitToPomise{
}
}
}
\end{lstlisting}
\end{lstlisting}
\section{\DSLSH}
In this thesis, we also created an alternative way of defining proposals and their respective transformations, this is done using JavaScript as it's own meta language for the definitions. The reason for creating a way of defining proposals using JavaScript is, it allows us to limit the amount of dependencies of the tool, since we no longer rely on \DSL, and it allows for more exploration in the future work of this project.
\DSLSH is less of an actual language, and more of a program API at the moment, it allows for defining proposals purely in JavaScript objects, which is meant to allow a more modular way of using this idea. In \DSLSH you define a \textit{prelude}, which is just a list of variable declarations that contain the type expression as a string for that given wildcard. This means we do not need to perform wildcard extraction when wanting to parse the templates used for matching and transformation.
\noindent\begin{minipage}{.45\textwidth}
\begin{lstlisting}
// Definition in JSTQL
proposal a{
case {
applicable to {
<<a:Expression>>
}
transform to {
() => <<a>>
}
}
}
\end{lstlisting}
\end{minipage}\hfil
\noindent\begin{minipage}{.45\textwidth}
\begin{lstlisting}[language={JavaScript}]
// Equivalent definition in JSTQL-SH
{
prelude: 'let a = "Expression"'`,
applicableTo: "a;",
transformTo: "() => a;"
}
\end{lstlisting}
\end{minipage}\hfil

View file

@ -354,4 +354,31 @@
urldate = {2024-05-24},
note = {[Online; accessed 24. May 2024]},
url = {https://tc39.es/ecma262/#sec-completion-record-specification-type}
}
@misc{Await,
title = {{ECMAScript{\ifmmode\circledR\else\textregistered\fi} 2025 Language Specification}},
year = {2024},
month = may,
urldate = {2024-05-24},
note = {[Online; accessed 24. May 2024]},
url = {https://tc39.es/ecma262/#await}
}
@misc{Promise,
title = {{ECMAScript{\ifmmode\circledR\else\textregistered\fi} 2025 Language Specification}},
year = {2024},
month = may,
urldate = {2024-05-24},
note = {[Online; accessed 24. May 2024]},
url = {https://tc39.es/ecma262/#sec-promise-objects}
}
@misc{TC39Process,
title = {{The TC39 Process}},
year = {2024},
month = apr,
urldate = {2024-05-24},
note = {[Online; accessed 24. May 2024]},
url = {https://tc39.es/process-document}
}