|
| 1 | +# Data Flow Facilitator for Machine Learning (dffml) |
| 2 | + |
| 3 | +[](https://travis-ci.org/intel/dffml) [](https://bestpractices.coreinfrastructure.org/projects/2594) |
| 4 | + |
| 5 | +DFFML provides APIs for dataset generation and storage, and model definition |
| 6 | +using any machine learning framework, from high level down to low level use is |
| 7 | +supported. |
| 8 | + |
| 9 | +The goal of DFFML is to build a community driven library of plugins for dataset |
| 10 | +generation and model definition. So that we as developers and researchers can |
| 11 | +quickly and easily plug and play various pieces of data with various model |
| 12 | +implementations. |
| 13 | + |
| 14 | +DFFML allows users to take advantage of Python's `asyncio` library in order to |
| 15 | +build applications which interact in deterministic ways with external data |
| 16 | +sources and syncs (think pub/sub, websockets, grpc streams). Writing with |
| 17 | +`asyncio` in the loop (huh-huh) makes testing MUCH MUCH EASIER. `asyncio` usage |
| 18 | +also means that when generating datasets, we can do everything concurrently (or |
| 19 | +in parallel if you want to us an executor) making generating a dataset from |
| 20 | +scratch very fast, and best of all, clean error handling if things go wrong. |
| 21 | + |
| 22 | +Here's a quick demo showing how DFFML can be used to train on the iris dataset. |
| 23 | +The more we build up the library of plugins (which anyone can maintain, they |
| 24 | +don't have to be contributed upstream unless you want to) the more variations on |
| 25 | +model implementations and feature data generators we all have to work with. |
| 26 | + |
| 27 | + |
| 28 | + |
| 29 | +Right now we've released a wrapper around the Tensorflow DNN estimator, and a |
| 30 | +set of feature generators which gather data from git repositories. |
| 31 | + |
| 32 | +## Installation |
| 33 | + |
| 34 | +DFFML currently should work with Python 3.6. However, only Python 3.7 is |
| 35 | +officially supported. This is because there are a lot of nice helper methods |
| 36 | +Python 3.7 implemented that we intend to use instead of re-implementing. |
| 37 | + |
| 38 | +```python |
| 39 | +python3.7 -m pip install -U dffml |
| 40 | +``` |
| 41 | + |
| 42 | +You can also install the Features for Git Version Control, and Models for |
| 43 | +Tensorflow Library all at once. |
| 44 | + |
| 45 | +- [DFFML Features for Git Version Control](feature/git/README.md) |
| 46 | +- [DFFML Models for Tensorflow Library](model/tensorflow/README.md) |
| 47 | + |
| 48 | +If you want a quick how to on the iris dataset head to the |
| 49 | +[DFFML Models for Tensorflow Library](model/tensorflow/README.md) repo. |
| 50 | + |
| 51 | +```python |
| 52 | +python3.7 -m pip install -U dffml[git,tensorflow] |
| 53 | +``` |
| 54 | + |
| 55 | +## Usage |
| 56 | + |
| 57 | +See [DFFML Models for Tensorflow Library](model/tensorflow/README.md) repo |
| 58 | +until documentation here is updated with a generic example. |
| 59 | + |
| 60 | +## Documentation |
| 61 | + |
| 62 | +Start with [Architechture](docs/ARCHITECHTURE.md). |
| 63 | + |
| 64 | +## License |
| 65 | + |
| 66 | +dffml is distributed under the [MIT License](LICENSE). |
| 67 | + |
| 68 | +## Legal |
| 69 | + |
| 70 | +> This software is subject to the U.S. Export Administration Regulations and |
| 71 | +> other U.S. law, and may not be exported or re-exported to certain countries |
| 72 | +> (Cuba, Iran, Crimea Region of Ukraine, North Korea, Sudan, and Syria) or to |
| 73 | +> persons or entities prohibited from receiving U.S. exports (including |
| 74 | +> Denied Parties, Specially Designated Nationals, and entities on the Bureau |
| 75 | +> of Export Administration Entity List or involved with missile technology or |
| 76 | +> nuclear, chemical or biological weapons). |
0 commit comments