Skip to content

Commit c1d1319

Browse files
patricijabreckopatricijabrecko
authored
Update README (#15)
This pull request significantly updates the `README.md` file for the `airdrop-shared` repository. It enhances documentation by restructuring content, adding detailed project setup instructions, and providing best practices for creating and managing DevRev snap-ins generated with Codeplain. Related work item: https://app.devrev.ai/devrev/works/ISS-190744
1 parent 19f87c5 commit c1d1319

File tree

1 file changed

+177
-32
lines changed

1 file changed

+177
-32
lines changed

‎README.md‎

Lines changed: 177 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,49 +1,194 @@
1-
# DevRev Repository for Snap-In Shared Files
1+
# Shared Files for Snap-in Generation with Codeplain
22

3-
This repository contains files that are shared between DevRev snapins. Example of related repositories:
3+
This repository (`airdrop-shared`) provides a centralized collection of files used as a common foundation for building DevRev snap-ins with the Codeplain generation tool. By sharing templates, scripts, and schemas, we can ensure consistency and accelerate development.
44

5-
- [devrev-trello-snapin](https://github.com/devrev/airdrop-trello-snap-in)
6-
- [devrev-asana-snapin](https://github.com/devrev/airdrop-asana-snap-in)
5+
### Related Snap-in Repositories
76

8-
Files in this repository are commonly shared between the above Snap-Ins, but you have the ability to override them for specific Snap-Ins.
7+
These repositories serve as examples of how `airdrop-shared` is used in practice:
98

10-
## Folder structure
9+
- [airdrop-trello-snap-in](https://github.com/devrev/airdrop-trello-snap-in)
10+
- [airdrop-wrike-snap-in](https://github.com/devrev/airdrop-wrike-snap-in)
1111

12-
This setup assumes this repository (`devrev-snapin-shared`) is cloned on the same level as the Snap-In repositories.
12+
-----
1313

14-
```bash
15-
❯ ls
16-
devrev-asana-snapin devrev-snapin-shared devrev-trello-snapin
14+
## Getting Started: Creating a New Snap-in
15+
16+
This guide walks you through the process of setting up a new snap-in project using the shared files from this repository.
17+
18+
### 1\. Prerequisites
19+
20+
Before you begin, ensure you have the following set up on your local machine.
21+
22+
#### Folder Structure
23+
24+
Your snap-in project directory must be a sibling to a clone of this `airdrop-shared` repository.
25+
26+
```sh
27+
# Your project layout should look like this
28+
.
29+
├── airdrop-shared/
30+
├── airdrop-trello-snap-in/
31+
├── airdrop-wrike-snap-in/
32+
└── your-new-snap-in/ # This is your new project
33+
```
34+
35+
#### Chef CLI
36+
37+
Download the newest version of `chef-cli` binary and place it in a known location on your local machine. Don't forget to make it executable.
38+
39+
- **Installation guide:** [Install Chef CLI](https://github.com/devrev/adaas-chef-cli/blob/main/README.md)
40+
41+
#### Codeplain Client
42+
43+
Clone the Codeplain client repository, which contains the code generation script.
44+
45+
- **Clone from:** [Codeplain-ai/plain2code\_client](https://github.com/Codeplain-ai/plain2code_client)
46+
47+
### 2\. Project Setup
48+
49+
In the root of your new snap-in project directory, create and configure the following files.
50+
51+
- **`manifest.yaml`**
52+
This file defines your snap-in's metadata. It is **not** generated automatically. You must create one tailored to your snap-in's needs. For guidance, refer to the [DevRev manifest documentation](https://developer.devrev.ai/public/airdrop/manifest).
53+
54+
- **`config.yaml`**
55+
This file configures the code generation process. You must include paths pointing to the cloned `airdrop-shared` repository. Other fields can be adjusted to assist with rendering.
56+
57+
*Example `config.yaml`:*
58+
59+
```yaml
60+
# Paths to the shared repository
61+
base-folder: ../airdrop-shared/base_folder
62+
template-dir: ../airdrop-shared
63+
unittests-script: ../airdrop-shared/scripts/run_unittests_jest.sh
64+
conformance-tests-script: ../airdrop-shared/scripts/run_devrev_snapin_conformance_tests.sh
65+
66+
# Other rendering options (see "Workflow & Best Practices" below)
67+
# render-from: 2.3.1
68+
# render-range: 2.3.1,2.3.2
69+
```
70+
71+
- **`.env`**
72+
Create a `.env` file to store sensitive information and local paths. At a minimum, it must contain the absolute path to the `chef-cli` binary. Add any other variables, such as API credentials, that your project requires.
73+
74+
*Example `.env` for a Wrike snap-in:*
75+
76+
```bash
77+
WRIKE_API_KEY=your_wrike_api_key
78+
WRIKE_SPACE_GID=your_wrike_space_gid
79+
80+
# Absolute path to the Chef CLI binary
81+
CHEF_CLI_PATH=/Users/yourname/path/to/chef-cli
82+
```
83+
84+
- **`devrev-<external-system>-snapin.plain`**
85+
This is the core specification file for your snap-in. It is highly recommended to copy the `.plain` file from an existing snap-in (like Trello or Wrike) and modify it. Start with a minimal set of functional requirements and expand as you go.
86+
87+
- Learn more about the specification language: [Plain Language Specification Guide](https://github.com/Codeplain-ai/plain2code_client/blob/main/Plain-language-specification.md)
88+
89+
- **`test_data/`** (Folder)
90+
Create this folder to store JSON data files that your Acceptance tests will use.
91+
92+
### 3\. Running Code Generation
93+
94+
#### Environment Preparation
95+
96+
1. Navigate to your cloned `plain2code_client` directory.
97+
2. Create and activate a Python virtual environment.
98+
3. Install the required dependencies:
99+
```sh
100+
pip install -r requirements.txt
101+
```
102+
4. Export the required environment variables.
103+
```sh
104+
# Your Codeplain API Key
105+
export CLAUDE_API_KEY="your_api_key"
106+
107+
# Absolute path to the cloned plain2code_client repository
108+
export PLAIN2CODE_RENDERER_DIR="/path/to/your/plain2code_client"
109+
```
110+
111+
#### Execute the Generation Script
112+
113+
From your snap-in's root directory, run the code generation script, pointing it to your `.plain` specification file.
114+
115+
```sh
116+
python $PLAIN2CODE_RENDERER_DIR/plain2code.py devrev-wrike-snapin.plain
17117
```
18118

19-
Make sure you have this setup locally as well.
119+
-----
120+
121+
## Development Workflow & Best Practices
122+
123+
- **Render Iteratively**: Instead of generating the entire snap-in at once, render one functional requirement (FR) at a time. This allows you to verify the generated code at each step. Use the `render-from` and `render-range` options in `config.yaml` to control this.
124+
125+
- `render-from: 2.3.1`: Renders from FR `2.3.1` to the end.
126+
- `render-range: 2.3.1,2.3.2`: Renders only FRs between `2.3.1` and `2.3.2` (inclusive).
127+
128+
> **Warning:** Regenerating a range of FRs will invalidate any subsequently generated requirements. For example, if you regenerate `2.3.1`-`2.3.2`, you will at some point also need to regenerate `2.3.3`.
129+
130+
- **Use Acceptance Tests for Complex Logic**: When a functional requirement becomes too complex to describe in detail or you're struggling with writing the FR, you can add an `Acceptance Tests` block directly in the `.plain` file. This helps guide the code generation model with a concrete example of the expected outcome.
131+
132+
*Example in a `.plain` file:*
133+
134+
```markdown
135+
- If "event_type" equals "EXTRACTION_DATA_START" The Extraction Function should:
136+
- push The Fetched Contacts to the repository named 'users'
137+
- push The Fetched Tasks to the repository designated for 'tasks' data
138+
(but make sure that a single "EXTRACTION_DATA_DONE" event is emitted)
139+
140+
***Acceptance Tests:***
141+
142+
- Test The Extraction Function using the resource [data_extraction_test.json](test_data/data_extraction_test.json). Test is successful if The Callback Server receives from DevRev a **single** event with "event_type" that equals "EXTRACTION_DATA_DONE". The test must not send event directly to The Callback Server.
143+
```
144+
145+
- **Debug with Conformance Tests**: If you encounter issues while rendering a specific functional requirement, you can run its associated conformance test individually. The `conformance_tests/conformance_tests.json` file maps FRs to their test names.
146+
147+
1. Identify the FR that is causing an issue.
148+
2. Look up the corresponding test name in `conformance_tests.json`.
149+
3. Run the specific test using the following command:
150+
151+
<!-- end list -->
152+
153+
```bash
154+
sh ../airdrop-shared/scripts/run_devrev_snapin_conformance_tests.sh build conformance_tests/<conformance-test-name>
155+
```
156+
157+
- **Use a Test Account**: Code generation is more accurate and successful when it can reference real-world data structures. Set up a test account in the target external system and populate it with representative data.
158+
159+
- **Handling Interruptions**: Code generation can take a long time. If your internet connection drops or the process is otherwise interrupted, it may appear to freeze.
160+
161+
1. Cancel the running Python script (`Ctrl+C`).
162+
2. Open `config.yaml` and set the `render-from` option to the ID of the last successfully completed functional requirement.
163+
3. Restart the generation script.
164+
165+
-----
166+
167+
## Repository Contents (`airdrop-shared`)
20168

21-
## Additional required files
169+
This repository contains the following shared components:
22170

23-
### chef-cli
171+
- `base_folder/`: A starter template for a new snap-in, adapted from [devrev/airdrop-template](https://github.com/devrev/airdrop-template).
172+
- `devrev-snapin-template.plain`: A general Plain Language specification that serves as a base template for all snap-ins.
173+
- **Resource Files**:
174+
- `test_data/`: Contains shared JSON test data files that are referenced in `.plain` specifications.
175+
- `external_domain_metadata_schema.json`: The JSON schema for external domain metadata.
176+
- **Scripts**:
177+
- `run_unittests_jest.sh`: A reusable script for running the snap-in's unit tests.
178+
- `run_devrev_snapin_conformance_tests.sh`: A reusable script for running the snap-in's conformance tests.
179+
- **Auxiliary Files**:
180+
- `jest.setup.js`, `mock_callback_server.py`: Helper files required by the unit test and conformance test scripts.
24181

25-
You must download locally to this folder chef-cli binary available at https://github.com/devrev/adaas-chef-cli/releases.
182+
### How to Override Shared Files
26183

27-
The chef-cli must be version 0.7.1 or newer.
184+
You may want to use your own custom specification or test files instead of the ones given in the `airdrop-shared` repository - for example if you'd like to write your own `devrev-snapin-template.plain`.
28185

29-
## The Shared Files
186+
Your local snap-in repository can override any file from this shared repository. Files in your snap-in's project directory always take precedence.
30187

31-
- Base folder: (`base_folder/`) - This folder represents starting point for the Snap-In. It's adjusted based on our needs from https://github.com/devrev/airdrop-template.
32-
- DevRev Plain Snap-In template (`devrev-snapin-template.plain`).
33-
- Resource files:
34-
- `test_data/` - This folder represents test data JSON files, referenced in the `.plain` files.
35-
- `external_domain_metadata_schema.json` - This file represents the external domain metadata schema.
36-
- Scripts:
37-
- `run_unittests_jest.sh` - Script for running unit tests for the Snap-In.
38-
- `run_devrev_snapin_conformance_tests.sh` - Script for running conformance tests for the Snap-In.
39-
- Auxiliary files:
40-
- `jest.setup.js`, `mock_callback_server.py` - Auxiliary files for unittest and conformance test scripts.
188+
- **`.plain` Templates**: To override `devrev-snapin-template.plain`, simply create a file with the same name in the root of your snap-in project.
41189

42-
## How to Override the Shared Files for Specific Snap-Ins
190+
- **Resource Files**: To override a resource file, create a new file at the **same relative path** within your snap-in project.
43191

44-
You have the ability to override the contents of the shared repository for your specific Snap-In. Files in the directory containting your plain source file will always have bigger preference than files from the shared folder.
192+
- *Example*: To override `airdrop-shared/test_data/data_extraction_check.json`, create the file `your-new-snap-in/test_data/data_extraction_check.json`.
45193

46-
- `devrev-snapin-template.plain` - In the workspace of your Plain source file (e.g. `devrev-trello-snapin`), define your custom `devrev-snapin-template.plain` file.
47-
- Resource files: You can override the contents resources by attaching a resource file to the same location relative to your plain source file.
48-
Example: If you have a resource file `test_data/data_extraction_check.json` in the shared repository, you can override it by attaching a resource file `test_data/data_extraction_check.json` to the same location relative to your plain source file.
49-
- Scripts and auxiliary files: You can copy+paste the script to your plain source file repository and adjust it to your needs.
194+
- **Scripts & Auxiliary Files**: To customize a script, copy it from `airdrop-shared/scripts/` into your snap-in's repository and modify it as needed. Remember to update the corresponding path in your `config.yaml`.

0 commit comments

Comments
 (0)