master/chapter/ch2.tex

\chapter{Background}

\section{Technical Committee 39}

Technical Committee 39  is the committee which maintains ECMA-262~\cite{ecma262}, the language standard for ECMAScript, and other related standards. They develop this standard following the TC39 process~\cite{TC39Process} for standard extension. 

Technical Committee 39 (abbreviated as TC39) is a group within ECMA international, whose main goal is to develop the language standard for ECMAScript (JavaScript) and other related standards. These related standards include: ECMA-402, the internalization API of ECMA-262, ECMA-404, the standard for JSON, ECMA-414, the ECMAScript specification suite standard. The members of the committee are representatives from various companies, academic institutions, and various other organizations from all across the world interested in developing the ECMAScript language. The members are usually people working wit JavaScript engines, tooling surrounding JavaScript, and other sections related to the JavaScript language.  


\subsection{ECMA-262 Proposals}

This section will contain what is a proposal, and how proposals are developed in TC39 for the ECMA-262 language standard. 


A proposal in this context is a suggested change to the ECMA-262 language standard. These additions to the standard have to solve some form of problem with the current version of ECMAScript. Such problems can come in many forms, and can apply to any part of the language. A problem can be, features that are not present in the language, inconsistent parts of the language, simplification of common patterns, etc etc. The proposal development process is defined in the document TC39 Process. 
\subsection*{TC39 Process}

The TC39 process~\cite{TC39Process}, is a process document describing how the extension ECMA-262 is performed. A suggested change to the ECMA-262 standard is in the form of a \emph{proposal}. This process documents describes the stages a proposal has to pass through to be accepted into the ECMA-262 standard. 

Stage 0 consists if ideation. The purpose of this stage is to allow for exploration and ideation around what part of the current version of ECMAScript can be improved, and then define a problem space for the committee to focus. 

Stage 1, is the point the committee has started taking the suggested addition and will consider it. The are several requirements to enter this stage: A champion has to be identified, a champion is a member TC39 who is responsible for the proposal. A rough outline of the problem, and a general shape of a solution. There has to have been discussion around key algorithms, abstractions and semantics of the proposal. Potential implementation challenges and cross-cutting concerns have to have been identified. All these described requirements have to be captured in a public repository. Once all these requirements are met, a proposal is accepted into stage 1. During this stage, the committee will work on designing a solution, and resolve any cross-cutting concerns discovered. 

Stage 2, a preferred solution has been identified. Requirements for a proposal to enter this stage: All high level APIs and syntax have to be described in the proposal document. Illustrative examples of usage created. An initial specification text have to be created. In this stage, the solution identified have to be refined, minor details ironed out, and experimental implementations will be created. 

Stage 2.7, the proposal is principally approved, and has to be tested and validated. To enter this stage, the major sections of the proposal have to be complete. The specification text is finished, and all reviewers of the specification have approved. Once a proposal has entered this stage, testing and validation will be performed. This is done through the prototype implementations created in stage 2, and all features of the proposal is validated. 

Stage 3, proposal is recommended for implementation. Once a proposal has been sufficiently tested and verified, it is moved to stage 3. During stage 3, the proposal is implemented in all major engines. During this stage, the proposal is tested for web compatibility issues, or integration issues in the major JavaScript engines. 

Stage 4, the proposal is completed and included in the standard. 

\section{AST and Babel}

\subsection*{Abstract Syntax Tree}

Abstract Syntax Trees (AST) is one of the most common representations used to represent source code of programming languages~\cite{AST3}. It allows for the storage of structured text into a tree, this is used in applications that work with source code, as source code is a form of structured text.   

\subsection*{Babel}

Babel is a JavaScript toolchain, its main usage is converting ECMASCript 2015 and newer into backwards older versions of JavaScript. The conversion to older versions is done to increase compatibility of JavaScript in older environments such as older browsers. 

Babel has a suite of libraries used to work with JavaScript source code, each library relies on Babels AST definition~\cite{BabelAST}. The AST specification Babel uses tries to stay as true to the ECMAScript standard as possible~\cite{BabelSpecCompliant}, which has made it a recommended parser to use for proposal transpiler implementations~\cite{TC39RecommendBabel}. A simple example of how source code parsed into an AST with Babel looks like can be seen in Figure~\ref{ex:srcToAST}.

\begin{figure}[H]
\noindent\begin{minipage}{.45\textwidth}
\begin{lstlisting}[language={JavaScript}]
let name = 100;
\end{lstlisting}
\end{minipage}\hfil
\noindent\begin{minipage}{.45\textwidth}
\begin{center}
\begin{tikzpicture}[
    squarednode/.style={rectangle, draw=red!60, fill=red!5, very thick, minimum size=2mm}, node distance=10mm and 5mm
]
\node[squarednode] (VarDecl)        {VariableDeclaration};
\node[squarednode] (VarDeclarator)  [below=of VarDecl] {VariableDeclarator};
\node[squarednode] (id)             [below left= 10mm and -10mm of VarDeclarator] {Identifier};
\node[squarednode] (init)           [below right= 10mm and -10mm of VarDeclarator] {NumericLiteral};

\draw[->] (VarDecl.south) -- (VarDeclarator.north);
\draw[->] (VarDeclarator.south) -- (id.north);
\draw[->] (VarDeclarator.south) -- (init.north);
\end{tikzpicture}
\end{center}
\end{minipage}\hfil
\caption{\label{ex:srcToAST} Example of source code parsed to Babel AST}
\end{figure}




Babel's mission is to transpile newer version of JavaScript into older versions that are more compatible with different environments. It has a rich plugin system to allow a myriad of features to be enabled or disabled. This makes the parser very versatile to fit different ways of working with JavaScript source code. 

Babels main feature is \texttt{@babel/parse}~\cite{BabelParser} and its plugins. This library allows parsing of JavaScript experimental features. These features are usually proposals that are under development by TC39, and the development of these plugins are a part of the proposal deliberation process. Some examples of proposals that were first  supported by Babels plugin system are "Do Expression"~\cite{Proposal:DoProposal} and "Pipeline"~\cite{Pipeline}. 


\section{Source code Querying}

Source code querying is a technique used to extract information from source code. Since source code is just structured text, one can treat it as a database, allowing for queries to be written and perform operations on that data. 

Source code Querying enables developers to search through their code for specific patterns based on a query written. These queries can come in many different forms. Such as SQL-like queries, structured queries, 

This kind of querying is very useful for programs such as Integrated Development Environments, source code transformations, and other tasks which require searching through code. An example of source code querying being used in an IDE is Jetbrains structural search and replace~\cite{StructuralSearchAndReplaceJetbrains}. 

\section{Domain Specific languages}


\subsection*{Language Workbenches}

One part of creating a software language is the tooling for that language. Most modern software languages are backed by some form of tooling, one important tool for a language is the language server. This is created to allow for features such as syntax highlighting, auto completion, error checking et.al. When creating a software language, generating tooling to support this features can be done using a \emph{language workbench}~\cite{LanguageWorkbench}. The language in question is specified within the language workbench, using a grammar. That grammar is then used to generate the tooling of the language. Many such language workbenches exist, such as Langium~\cite{Langium}, Xtext~\cite{Xtext}, Jetbrains MPS, and Racket.