Skip to content

Commit 240a9fe

Browse files
committed
Added Github Actions example
1 parent 44fe5bb commit 240a9fe

File tree

6 files changed

+156
-0
lines changed

6 files changed

+156
-0
lines changed

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ Detailed information on the CLI tool sub-commands and arguments can be found in
4646
- [Dialogflow endpoint configuration](docs/DialogflowEndpointConfiguration.md)
4747
- [Configuring LUIS CI/CD with Azure Pipelines](docs/AzurePipelines.md)
4848
- [Extending the CLI to new NLU providers](docs/CliExtensions.md)
49+
- [GitHub Actions workflow using NLU.DevOps cli tool](docs/NLUGitHubActions.md)
4950

5051
## Contributing
5152

docs/NLUGitHubActions.md

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
## GitHub Actions workflow using NLU.DevOps CLI tool
2+
3+
This document covers setting up a CI pipeline for LUIS using GitHub Actions and NLU.DevOps.
4+
The workflow will be the following: 
5+
1. create, train, publish the LUIS model using sample utterances
6+
2. send a test set to created LUIS model
7+
3. evaluate model by comparing results received from LUIS with expected values
8+
4. delete the model from the portal
9+
10+
Supply a name for the action and set it up to trigger on pull requests.
11+
12+
```
13+
name: CINLU
14+
15+
on: [pull_request]
16+
```
17+
18+
1. Install NLU.DevOps CLI tool on GitHub agent.
19+
20+
```
21+
- name: Install dotnet-nlu
22+
run: dotnet tool install -g dotnet-nlu
23+
```
24+
For Ubuntu agents, you need to prepend a directory to the system PATH variable for all subsequent actions in the current job to make sure that CLI tool works. More about this command [here](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/development-tools-for-github-actions#add-a-system-path-add-path).
25+
26+
```
27+
- name: Path
28+
run: echo "::add-path::$HOME/.dotnet/tools"
29+
```
30+
We use [utterances.json](../models/utterances.json) for training. You can replace this file with another file that consists of intents, utterances, entities that you need for your own model.
31+
More about the format of this file [here](https://github.com/microsoft/NLU.DevOps/blob/master/docs/GenericUtterances.md).
32+
To train your model we should add the following:
33+
34+
```
35+
- name: Train Luis model
36+
run: dotnet nlu train -s luis -u utterances.json --save-appsettings
37+
env:
38+
luisAuthoringRegion: ${{ secrets.luisAuthoringRegion }}
39+
luisAuthoringKey: ${{ secrets.luisAuthoringKey }}
40+
```
41+
42+
More about the command [here](https://github.com/microsoft/NLU.DevOps/blob/master/docs/Train.md)
43+
Before you push to the repo, you need to add credentials (at least luisAuthoringKey and luisAuthoringRegion are required) to your GitHub Secrets. For example,
44+
![credentials](./images/credentials.png)
45+
46+
Check out the [LUIS docs](https://docs.microsoft.com/en-us/azure/cognitive-services/luis/luis-how-to-azure-subscription) for more information on where to find your authoring or runtime keys.
47+
48+
2. To test LUIS model let's use [utterancesTest.json](../models/utterancesTest.json) file.
49+
We can save the result in results.json. During the training step, we may have created a new LUIS application. By using the `--save-appsettings` flag, the LUIS application ID is stored locally in a settings file that is picked up by subsequent NLU.DevOps CLI commands.
50+
51+
Yaml may look like that:
52+
```
53+
- name: Test Luis model
54+
run: dotnet nlu test -s luis -u utterancesTest.json -o results.json
55+
env:
56+
luisAuthoringRegion: ${{ secrets.luisAuthoringRegion }}
57+
luisAuthoringKey: ${{ secrets.luisAuthoringKey }}
58+
```
59+
3. We use the `compare` command from the NLU.DevOps CLI to evaluate the results. The expected intents and entities in this case are given in the `utterancesTest.json` file, while the results predicted by the LUIS model are in the `results.json` file.
60+
61+
```
62+
- name: Compare Luis model
63+
run: dotnet nlu compare -e utterancesTest.json -a results.json
64+
```
65+
66+
If you open your GitHub workflow run step for this command in the console, you can see something similar to
67+
![compareResults](./images/compareResults.png)
68+
69+
4. When you work on several hypotheses, sometimes you need only to get results and you don't want to keep the model. It is possible to delete the model in the same pipeline after you get results.
70+
```
71+
- name: Delete Luis model
72+
run: dotnet nlu clean -s luis
73+
env:
74+
luisAuthoringRegion: ${{ secrets.luisAuthoringRegion }}
75+
luisAuthoringKey: ${{ secrets.luisAuthoringKey }}
76+
```
77+
78+
You can find GitHub Action workflow yaml file [here](../pipelines/.github/workflows/nlugithub.yml).

docs/images/compareResults.PNG

6.59 KB
Loading

docs/images/credentials.PNG

38.2 KB
Loading

models/utterancesTest.json

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
[
2+
{
3+
"text": "start playing music",
4+
"intent": "PlayMusic"
5+
},
6+
{
7+
"text": "play music",
8+
"intent": "PlayMusic"
9+
},
10+
{
11+
"text": "listen to hip hop",
12+
"intent": "PlayMusic"
13+
},
14+
{
15+
"text": "is it cold out",
16+
"intent": "None"
17+
},
18+
{
19+
"text": "how many days until Christmas",
20+
"intent": "None"
21+
},
22+
{
23+
"text": "what's the weather like",
24+
"intent": "None"
25+
}
26+
]
27+
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
name: CINLU
2+
3+
on: [pull_request]
4+
5+
jobs:
6+
build:
7+
name: NLU workflow
8+
runs-on: ubuntu-latest
9+
10+
steps:
11+
- uses: actions/checkout@v1
12+
13+
- name: Setup .NET Core
14+
uses: actions/setup-dotnet@v1
15+
with:
16+
dotnet-version: 2.2.108
17+
18+
- name: Install dotnet-nlu
19+
run: dotnet tool install -g dotnet-nlu
20+
21+
- name: Path
22+
run: echo "::add-path::$HOME/.dotnet/tools"
23+
24+
- name: Train Luis model
25+
run: dotnet nlu train -s luis -u utterances.json --save-appsettings
26+
env:
27+
luisAuthoringRegion: ${{ secrets.luisAuthoringRegion }}
28+
luisAuthoringKey: ${{ secrets.luisAuthoringKey }}
29+
30+
- name: Test Luis model
31+
run: dotnet nlu test -s luis -u utterancesTest.json -o results.json
32+
env:
33+
luisAuthoringRegion: ${{ secrets.luisAuthoringRegion }}
34+
luisAuthoringKey: ${{ secrets.luisAuthoringKey }}
35+
36+
- name: Compare Luis model
37+
run: dotnet nlu compare -e utterancesTest.json -a results.json
38+
39+
- name: Archive TestResult
40+
uses: actions/upload-artifact@v1
41+
with:
42+
name: TestResult
43+
path: TestResult.xml
44+
45+
- name: Delete Luis model
46+
run: dotnet nlu clean -s luis
47+
env:
48+
luisAuthoringRegion: ${{ secrets.luisAuthoringRegion }}
49+
luisAuthoringKey: ${{ secrets.luisAuthoringKey }}
50+

0 commit comments

Comments
 (0)