# Code AST
> Fast structural analysis of any programming language in Python
Programming Language Processing (PLP) brings the capabilities of modern NLP systems to the world of programming languages.
To achieve high performance PLP systems, existing methods often take advantage of the fully defined nature of programming languages. Especially the syntactical structure can be exploited to gain knowledge about programs.
**code.ast** provides easy access to the syntactic structure of a program. By relying on [tree-sitter](https://github.com/tree-sitter) as the back end, the parser supports fast parsing of variety of programming languages.
The goal of code.ast is to combine the efficiency and variety of languages supported by tree-sitter with the convenience of more native parsers (like [libcst](https://github.com/Instagram/LibCST)).
To achieve this, code.ast adds the features:
1. **Auto-loading:** Compile of source code parsers for any language supported by tree-sitter with a single keyword,
2. **Visitors:** Search the AST quickly,
3. **Transformers:** Transform source code easily by transforming the AST structure
## Installation
The package is tested under Python 3. It can be installed via:
```bash
pip install code-ast
```
## Quick start
code.ast can parse nearly any program code in a few lines of code:
```python
import code_ast
# Python
code_ast.ast(
'''
def my_func():
print("Hello World")
''',
lang = "python")
# Output:
# PythonCodeAST [0, 0] - [4, 4]
# module [1, 8] - [3, 4]
# function_definition [1, 8] - [2, 32]
# identifier [1, 12] - [1, 19]
# parameters [1, 19] - [1, 21]
# block [2, 12] - [2, 32]
# expression_statement [2, 12] - [2, 32]
# call [2, 12] - [2, 32]
# identifier [2, 12] - [2, 17]
# argument_list [2, 17] - [2, 32]
# string [2, 18] - [2, 31]
# Java
code_ast.ast(
'''
public class HelloWorld {
public static void main(String[] args){
System.out.println("Hello World");
}
}
''',
lang = "java")
# Output:
# JavaCodeAST [0, 0] - [7, 4]
# program [1, 0] - [6, 4]
# class_declaration [1, 0] - [5, 1]
# modifiers [1, 0] - [1, 6]
# identifier [1, 13] - [1, 23]
# class_body [1, 24] - [5, 1]
# method_declaration [2, 8] - [4, 9]
# ...
```
## Visitors
code.ast implements the visitor pattern to quickly traverse the AST structure:
```python
import code_ast
from code_ast import ASTVisitor
code = '''
def f(x, y):
return x + y
'''
# Count the number of identifiers
class IdentifierCounter(ASTVisitor):
def __init__(self):
self.count = 0
def visit_identifier(self, node):
self.count += 1
# Parse the AST and then visit it with our visitor
source_ast = code_ast.ast(code, lang = "python")
count_visitor = IdentifierCounter()
source_ast.visit(count_visitor)
count_visitor.count
# Output: 5
```
## Transformers
Transformers provide an easy way to transform source code. For example, in the following, we want to mirror each binary addition:
```python
import code_ast
from code_ast import ASTTransformer, FormattedUpdate, TreeUpdate
code = '''
def f(x, y):
return x + y + 0.5
'''
# Mirror binary operator on leave
class MirrorAddTransformer(ASTTransformer):
def leave_binary_operator(self, node):
if node.children[1].type == "+":
return FormattedUpdate(
" %s + %s",
[
TreeUpdate(node.children[2]),
TreeUpdate(node.children[0])
]
)
# Parse the AST and then visit it with our visitor
source_ast = code_ast.ast(code, lang = "python")
mirror_transformer = MirrorAddTransformer()
# Mirror transformer are initialized by running them as visitors
source_ast.visit(mirror_transformer)
# Transformer provide a minimal AST edit
mirror_transformer.edit()
# Output:
# module [2, 0] - [5, 0]
# function_definition [2, 0] - [3, 22]
# block [3, 4] - [3, 22]
# return_statement [3, 4] - [3, 22]
# binary_operator -> FormattedUpdate [3, 11] - [3, 22]
# binary_operator -> FormattedUpdate [3, 11] - [3, 16]
# And it can be used to directly transform the code
mirror_transformer.code()
# Output:
# def f(x, y):
# return 0.5 + y + x
```
## Project Info
The goal of this project is to provide developer in the
programming language processing community with easy
access to AST parsing. This is currently developed as a helper library for internal research projects. Therefore, it will only be updated
as needed.
Feel free to open an issue if anything unexpected
happens.
Distributed under the MIT license. See ``LICENSE`` for more information.
We thank the developer of [tree-sitter](https://tree-sitter.github.io/tree-sitter/) library. Without tree-sitter this project would not be possible.