Skip to content
This repository was archived by the owner on Jul 1, 2024. It is now read-only.

Commit 4857329

Browse files
royweiroywei
authored
Keras MXNet 2.1.6 Release (#101)
* Improve tests by designating dtype of sample data (keras-team#9834) * Document that "same" is inconsistent across backends with strides!=1 (keras-team#9629) * Document that `"same"` is inconsistent across backends with `strides` != 1 * Use "[here](...)" * keras-team#9642 Add kwarg and documentation for dilation_rate to SeparableConvs (keras-team#9844) * Add kwarg and documentation for dilation_rate to SeparableConvs * Fix pep8 complaint I forgot to check the style before committing. Pep8 was complaining about a missing whitespace after comma, now it's fixed. * fit/evaluate_generator supporting native tensors (keras-team#9816) Currently, `fit/evaluate_generator` don't support this case without this fix. But framework-native data tensors are already supported by `_fit_loop` and `_test_loop`. Signed-off-by: CUI Wei <ghostplant@qq.com> * Add h5py to dependencies * Fixed typo. (keras-team#9866) * Fix image_ocr.py example ValueError (keras-team#9869) * Fixed the NASNet issue. (keras-team#9865) * Fixed the NASNet issue. * Nasnet doesn't require flatten. * Updated documentation accordingly. * Removed generate dropout ones from recurrent. (keras-team#9892) * Removed generate dropout ones from recurrent. * Fixed index issue. * Fix `in_test_phase` of CNTK and Add its tests (keras-team#9902) * Fix dtype designation for `variable` of CNTK and Add its tests (keras-team#9903) * import `pydot`, improve error messages about `pydot` and GraphViz, bump to `pydot >= 1.2.4` (keras-team#9904) * REL: bump to `pydot >= 1.2.4` in `extras_require` * MAI: import pydot (as required in `extras_require`) * MAI: refine error messages for `pydot` and GraphViz distinguish between absence of `pydot` and failure to find the executables of GraphViz in the $PATH. * DEV: ignore `.pytest_cache` * Fix documentation of flow_from_directory() (keras-team#9910) The way the documentation is parsed for the Keras website made some lines of the documentation beginning with "Default:" look funny. Also changed the documentation of return value to be clear that it always returns a batch of images. * ModelCheckpoint: print previous best (keras-team#9911) * multi_gpu_model supporting legacy/fullCPU/fullGPU (keras-team#9638) Signed-off-by: CUI Wei <ghostplant@qq.com> * Fix `batch_dot` of Theano when `axes=0` (keras-team#9920) * Fix `batch_dot` of CNTK when `axes=None` (keras-team#9921) * Fix `batch_dot` of TensorFlow when `axes=None` (keras-team#9922) * Fix stateful metrics when passing dict to compile (keras-team#9894) * Added note to manually install h5py where needed (keras-team#9830) * Added notes to manually install h5py if needed * Added FAQ entry on h5py * deleted redundant remark about h5py * updated FAQ to reflect dependency change * fixed comment format to pass failing test * removed new trailing whitespaces * improved docstring format * reverted callbacks.py * fixed links in model.py * updated faq.py * link pointing to FAQ * Add support for `constants` in Bidirectional wrapper (keras-team#9260) * Add support fot `constants` in Bidirectional wrapper * Add more tests for Bidirectional wrapper * Fix `compute_mask` for Birectional with return_state=True Fix `compute_mask` to properly support `return_state` introduced in Birectional with keras-team#8977 * Add test for Bidirectional with unknown timestamps * Skip test for CNTK for unknown timestamps with Bidirectional * avoid override the input constant when need broadcast sequential axis on rnn's constant * Move _standardize_args to recurrent, remove duplication * Fix for Birectional when multiple masks are passed * Updated for TF 1.7 (keras-team#9937) * fix TimeSeriesGenerator glitch (keras-team#9899) * Added an error message for undefined shape on NASNet. (keras-team#9891) * Added an error message for undefined shape on NASNet. * Forgot that the message should be present only when loading imagenet weights. * Changed the message. * Fix PEP8 * Allow shift_range to be 1-D array-like or int (keras-team#8869) * Allow shift_range to be 1-D array-like or int * Add docstrings * Fix conflict resolution merge minor disaster * remove stray line from merge * Remove extra "tabs" * Exclude multi-gpu utils when reporting coverages (keras-team#9942) * Make conv_invalid_use and pooling_invalid_use efficient (keras-team#9944) * Chenta/cntk bn (keras-team#9952) * fix cntk static learning phase issue; add a test * fix code style;add more comments * add boolean support * fix code style issue * Immigrate reference operations to a separate module (keras-team#9948) * Add MXNet Backend (#59) * Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1) * add activation functions * add activation functions * fix some legacy * fix some legacy * cross entropy * cross entropy * fix name scoping introduced in 2.0 * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * remove the logic for hacking RNN * add pooling with utils * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * mxnet_backend graph fix, layer support (#3) * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * add pooling with utils * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * remove the logic for hacking RNN * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * Keras function not working is a known issue, add skip in the test * fix random_uniform/constant * fix legacy randomize methods * Fix MXNet backend operator bugs. Enabled Keras backend tests * add bias * Add Amazon copyrights to License (#6) * fix * fix * fix backend for mlp * fix context management, add optimizers * minor change * undo changes on example * fix eval * minor cleanup * fix some property usage * fixing AlphaDroupout, not finished yet * add mx model instantiate * modifies training model construct logic, fix some tests. fix reshape layer. * minor fix * fix bias_add * more fix on Dense and bias_add * In progress commit * fix comment * small fix * remove pytest.skip in conv3d. But it failed with theano backend in my workspace though. * Add conv2d and in_topk operator for mxnet backend (#11) * Skip BatchDot tests for Theano backend. (#12) * BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14) * Fix Conv2d shape issues and enable Conv2D UTs * Remove redundant mxnet only unit tests * Adding batch_dot, remove deconv, code comments and cleanup * Remove buggy conv1d implementation * Fix CR comments. Fix lint check issues * Move mxnet specific code from keras engine to mxnet_backend. (#15) * Move MXNet optimizers from keras optimizers to mxnet backend (#16) * Fix bug in reshape. Minor rename to avoid local conflicts * Bug fixes and enable/skip all Keras tests for mxnet backend (#21) * test results - 374 passed, 235 skipped in 114.44 seconds * fix/skip keras tests - tests/integration_tests, tests/keras/applications * fix/skip keras tests - tests/keras/engine/test_topology * fix/skip keras tests - tests/keras/engine/test_training * fix/skip keras tests - tests/keras/legacy/ * fix/skip keras tests - tests/keras/preprocessing * fix/skip keras tests - tests/keras/utils/ * Fix CR comments * Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test * Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization * Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test * Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving * Fix mxnet backend multi-gpu training (#31) Fixing bug for mxnet backend to use multiple gpus. * Fix performance issue - Batchnormalization, Conv operator (#35) * Fix default axis for batchnorm layer for channels_first data_format * Performance improvement by avoiding kernel transpose in conv operation for channels_first format * Fix model - architecture, weights and both, load and save. (#36) * Prepare initial version of mxnet related documentation in keras (#38) * Skip failing unit tests for unsupported functionality in mxnet backend * Fix pep tests reported by CI * Use pytest module skip, revert kernel_shape logic * remove data_format param from bias_add API * Allow Predict() without compile for mxnet backend and enable tests. contributor - roywei@ * Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance * Conv3d() operator implementation for Keras2.0 using MXNet backend (#40) * conv3d implementation for keras2.0 as MXNet backend * conv3d implementation/testing for keras2.0 using MXNet backend * keeping -n option in pytest.ini file * fixed comments given by Sandeep * Add Conv1D support for MXNet backend (#44) * Add Conv1D support for MXNet backend * Fix CR comments * Conv2d transpose (#47) * add conv2d_transpose * conv2d transpose for both channels, enabled test case * add detailed comments and examples, fix style issue * enable test case in topology * Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48) * Fix conv kernel shape bug for TF backend. (#50) * Add support for keras multi_gpu_model() API with MXNet backend (#49) * Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine * Fix typo * Add SAME padding mode support for pooling operator. (#51) * Add rnn() operator for MXNet backend with unrolling and masking feature (#46) * Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators. * Modified comments * Added comments to a method * Enable categorical crossentropy testcases and made minor changes * Modified message * nit * Added detail description of handling variable length input in RNN * Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn() * Adamax and NAdam optimizer for MXNet backend (#54) * Add Adamax optimizer for MXNet backend * Fix lr and adamax params * Add Nadam optimizer for mxnet backend * Add Conv3d transpose (#52) * conv3d tranpose, enabled test case * update kernel shape * replace conv2d_transpse conv3d_transpose with convnd_transpose * update value errors with MXNet Backend info, fix typo * add check for conv3d transpose only supports gpu with cudnn * update context check * diable conv3d transpose test * fix typo in comment * Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1) * add activation functions * add activation functions * fix some legacy * fix some legacy * cross entropy * cross entropy * fix name scoping introduced in 2.0 * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * remove the logic for hacking RNN * add pooling with utils * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * mxnet_backend graph fix, layer support (#3) * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * add pooling with utils * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * remove the logic for hacking RNN * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * Keras function not working is a known issue, add skip in the test * fix random_uniform/constant * fix legacy randomize methods * Fix MXNet backend operator bugs. Enabled Keras backend tests * add bias * Add Amazon copyrights to License (#6) * fix * fix * fix backend for mlp * fix context management, add optimizers * minor change * undo changes on example * fix eval * minor cleanup * fix some property usage * fixing AlphaDroupout, not finished yet * add mx model instantiate * modifies training model construct logic, fix some tests. fix reshape layer. * minor fix * fix bias_add * more fix on Dense and bias_add * In progress commit * fix comment * small fix * remove pytest.skip in conv3d. But it failed with theano backend in my workspace though. * Add conv2d and in_topk operator for mxnet backend (#11) * Skip BatchDot tests for Theano backend. (#12) * BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14) * Fix Conv2d shape issues and enable Conv2D UTs * Remove redundant mxnet only unit tests * Adding batch_dot, remove deconv, code comments and cleanup * Remove buggy conv1d implementation * Fix CR comments. Fix lint check issues * Move mxnet specific code from keras engine to mxnet_backend. (#15) * Move MXNet optimizers from keras optimizers to mxnet backend (#16) * Fix bug in reshape. Minor rename to avoid local conflicts * Bug fixes and enable/skip all Keras tests for mxnet backend (#21) * test results - 374 passed, 235 skipped in 114.44 seconds * fix/skip keras tests - tests/integration_tests, tests/keras/applications * fix/skip keras tests - tests/keras/engine/test_topology * fix/skip keras tests - tests/keras/engine/test_training * fix/skip keras tests - tests/keras/legacy/ * fix/skip keras tests - tests/keras/preprocessing * fix/skip keras tests - tests/keras/utils/ * Fix CR comments * Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test * Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization * Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test * Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving * Fix mxnet backend multi-gpu training (#31) Fixing bug for mxnet backend to use multiple gpus. * Fix performance issue - Batchnormalization, Conv operator (#35) * Fix default axis for batchnorm layer for channels_first data_format * Performance improvement by avoiding kernel transpose in conv operation for channels_first format * Fix model - architecture, weights and both, load and save. (#36) * Prepare initial version of mxnet related documentation in keras (#38) * Skip failing unit tests for unsupported functionality in mxnet backend * Fix pep tests reported by CI * Use pytest module skip, revert kernel_shape logic * remove data_format param from bias_add API * Allow Predict() without compile for mxnet backend and enable tests. contributor - roywei@ * Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance * Conv3d() operator implementation for Keras2.0 using MXNet backend (#40) * conv3d implementation for keras2.0 as MXNet backend * conv3d implementation/testing for keras2.0 using MXNet backend * keeping -n option in pytest.ini file * fixed comments given by Sandeep * Add Conv1D support for MXNet backend (#44) * Add Conv1D support for MXNet backend * Fix CR comments * Conv2d transpose (#47) * add conv2d_transpose * conv2d transpose for both channels, enabled test case * add detailed comments and examples, fix style issue * enable test case in topology * Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48) * Fix conv kernel shape bug for TF backend. (#50) * Add support for keras multi_gpu_model() API with MXNet backend (#49) * Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine * Fix typo * Add SAME padding mode support for pooling operator. (#51) * Add rnn() operator for MXNet backend with unrolling and masking feature (#46) * Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators. * Modified comments * Added comments to a method * Enable categorical crossentropy testcases and made minor changes * Modified message * nit * Added detail description of handling variable length input in RNN * Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn() * Adamax and NAdam optimizer for MXNet backend (#54) * Add Adamax optimizer for MXNet backend * Fix lr and adamax params * Add Nadam optimizer for mxnet backend * Add Conv3d transpose (#52) * conv3d tranpose, enabled test case * update kernel shape * replace conv2d_transpse conv3d_transpose with convnd_transpose * update value errors with MXNet Backend info, fix typo * add check for conv3d transpose only supports gpu with cudnn * update context check * diable conv3d transpose test * fix typo in comment * Rebase to latest Keras - April 3, 2018 * Add build badges * Fix multi_gpu API bug for CPU. Fix PEP. (#64) * Fix multi_gpu API bug for CPU. Fix PEP. * fix embedding layer bug (#61) * fix embedding bug * addressed comments, enabled more test cases * add keras test * reduce line length * fix style, add blank lines * Benchmark (#55) * add conv2d_transpose * conv2d transpose for both channels, enabled test case * add detailed comments and examples, fix style issue * add benchmark scripts for resnet and imagenet data * combine scripts * fix args * fix num of gpus * update log * multi_gpu_model only support tf * add benchamrk scripts for synthetic data * update read me and scripts * add mxnet traing result table * update on readme * add cifar10 dataset and enable various resnet layers * fix compile for mxnet multiple gpu * update callbacks * update synthetic data script, add credits * undo new line * update readme, addressed pr comments * update readme * benchmark scripts style fix (#66) * style fix * remove unused import, fix line too long * adrressed pr comments * Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend (#65) * Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend * Modified comments * Addressed review comments and made the API more generic accross backends * Removed shape check * Modified comments * Added edge cases * moved helper method as nested * Added RNN benchmark scripts (#69) * Added RNN benchmark scripts * Fixed new line in bash script * Removed different backend code and modified comments * Removed spacing * Automated the wikiText2 download script * Added dataset_util functionality to have more flexible code * Added minor comments * modified minor comments * Fixed the multi-gpu context (#68) * Update benchmark result (#70) * update benchmark result * update result * simplify folder structure * add image result * add note * add note * rebase to latest Keras - April 20, 2018, fix bug and unit tests * Added detailed RNN results (#73) * Added detailed RNN results * Modified table content and added CUDA version * fix keras examples (#72) * fix auto encoder examples * update other examples * fix style and add ctc not implemented error * Added Detailed RNN results (#77) * Modified RNN benchmark document * Added minor comments * fixed broken image link * Added API to extract metrics from a test and also added epoch parameter (#78) * Add mxnet backend tutorial documents (#76) * add performance tips document * update warning * add docs from wiki * add initial multi gpu doc, simplified installation doc, fix benchmark doc typo * update install steps * add multi_gpu_model tutorial * Support exporting model as MXNet model (sym, params). (#80) * Support exporting model as MXNet model (sym, params). * Return data_names and data_shapes * add unit tests for mxnet model save API * Add test with LSTM layer for mxnet model save API * Add support for functional Model graphs in save_mxnet_model API * add multi gpu model example (#85) * add multi gpu model * specify param name * Add additional logging for cnn benchmarks (#89) * add extra logging * add logging for cnn synthetic * fix log name * fix file name * Log RNN benchmark results (#90) * Make benchmark result logging available in RNN scripts * Make log file name consistent across CNN and RNN benchmarks * fix pytest errors (#93) * Cherry pick keras-team/keras 2.1.6 missing 3 commits into awslabs/keras-apache-mxnet (#96) * update multi_gpu api in benchmark scripts (#95) * update multi_gpu * update logging * fix logging * fix logging * fix speed format * remove learning rate log * Revamp keras-mxnet docs (#82) * Update main README and move mxnet_backend_docs under docs * revisit installation mxnet backend docs * revisit multi_gpu_training mxnet backend docs * revisit performance_guide mxnet backend docs * revisit using rnn with mxnet backend in mxnet backend docs * add save_mxnet_model tutorials in mxnet backend docs * Fixing review comments from aaron * Resolve CR comments on save_mxnet_model tutorial * Fix broken links, update tutorial links in the mxnet_backend code * revamp benchmark results readme * Benchmark results README page revamp * Add library versions * Remove too detailed benchmark results. Summarize in README * Get back detailed results document * Remove experiemental RNN benchmarks from README * addressed review comments on benchmark results * Set latest stable dependency of h5py to avoid warnings * Update CNN benchmark result (#97) * update benchmark numbers * update number * update result * Update RNN benchmark results (#98) * Fix pep failures * Add 8 GPUs RNN benchmark results * remove checking data format (#102) * update imagenet result (#103)
1 parent 865d005 commit 4857329

File tree

9 files changed

+68
-75
lines changed

9 files changed

+68
-75
lines changed

‎benchmark/README.md‎

Lines changed: 22 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -62,28 +62,28 @@ NOTE:
6262

6363
| Instance Type | GPUs | Batch Size | Keras-MXNet (img/sec) | Keras-TensorFlow (img/sec) |
6464
|---|---|---|---|---|
65-
| P3.8X Large | 1 | 32 | 165 | 50 |
66-
| P3.8X Large | 4 | 128 | 538 | 162 |
67-
| P3.16X Large | 8 | 256 | 728 | 212 |
65+
| P3.8X Large | 1 | 32 | 135 | 52 |
66+
| P3.8X Large | 4 | 128 | 536 | 162 |
67+
| P3.16X Large | 8 | 256 | 722 | 211 |
6868

6969
#### ResNet50-Synthetic Data
7070

7171
| Instance Type | GPUs | Batch Size | Keras-MXNet (img/sec) | Keras-TensorFlow (img/sec) |
7272
|---|---|---|---|---|
73-
| C5.18X Large | 0 | 32 | 9 | 4 |
74-
| P3.8X Large | 1 | 32 | 229 | 164 |
75-
| P3.8X Large | 4 | 128 | 728 | 409 |
76-
| P3.16X Large | 8 | 256 | 963 | 164 |
73+
| C5.18X Large | 0 | 32 | 13 | 4 |
74+
| P3.8X Large | 1 | 32 | 194 | 184 |
75+
| P3.8X Large | 4 | 128 | 764 | 393 |
76+
| P3.16X Large | 8 | 256 | 1068 | 261 |
7777

7878

7979
#### ResNet50-CIFAR10
8080

8181
| Instance Type | GPUs | Batch Size | Keras-MXNet (img/sec) | Keras-TensorFlow (img/sec) |
8282
|---|---|---|---|---|
8383
| C5.18X Large | 0 | 32 | 87 | 59 |
84-
| P3.8X Large | 1 | 32 | TBD | TBD |
85-
| P3.8X Large | 4 | 128 | 1792 | 1020 |
86-
| P3.16X Large | 8 | 256 | 1618 | 962 |
84+
| P3.8X Large | 1 | 32 | 831 | 509 |
85+
| P3.8X Large | 4 | 128 | 1783 | 699 |
86+
| P3.16X Large | 8 | 256 | 1680 | 435 |
8787

8888

8989
You can see more benchmark experiments with different instance types, batch_size and other parameters in [detailed CNN
@@ -210,10 +210,10 @@ For MXNet backend benchmarks:
210210

211211
For TensorFlow backend benchmarks:
212212
```
213-
$ sh run_tf_backend.sh cpu_config lstm_nietzsche False 20 # For CPU Benchmarks
214-
$ sh run_tf_backend.sh gpu_config lstm_nietzsche False 20 # For 1 GPU Benchmarks
215-
$ sh run_tf_backend.sh 4_gpu_config lstm_nietzsche False 20 # For 4 GPU Benchmarks
216-
$ sh run_tf_backend.sh 8_gpu_config lstm_nietzsche False 20 # For 8 GPU Benchmarks
213+
$ sh run_tf_backend.sh cpu_config lstm_nietzsche False 10 # For CPU Benchmarks
214+
$ sh run_tf_backend.sh gpu_config lstm_nietzsche False 10 # For 1 GPU Benchmarks
215+
$ sh run_tf_backend.sh 4_gpu_config lstm_nietzsche False 10 # For 4 GPU Benchmarks
216+
$ sh run_tf_backend.sh 8_gpu_config lstm_nietzsche False 10 # For 8 GPU Benchmarks
217217
```
218218

219219
#### LSTM-WikiText2
@@ -230,10 +230,10 @@ For MXNet backend benchmarks:
230230

231231
For TensorFlow backend benchmarks:
232232
```
233-
$ sh run_tf_backend.sh cpu_config lstm_wikitext2 False 20 # For CPU Benchmarks
234-
$ sh run_tf_backend.sh gpu_config lstm_wikitext2 False 20 # For 1 GPU Benchmarks
235-
$ sh run_tf_backend.sh 4_gpu_config lstm_wikitext2 False 20 # For 4 GPU Benchmarks
236-
$ sh run_tf_backend.sh 8_gpu_config lstm_wikitext2 False 20 # For 8 GPU Benchmarks
233+
$ sh run_tf_backend.sh cpu_config lstm_wikitext2 False 10 # For CPU Benchmarks
234+
$ sh run_tf_backend.sh gpu_config lstm_wikitext2 False 10 # For 1 GPU Benchmarks
235+
$ sh run_tf_backend.sh 4_gpu_config lstm_wikitext2 False 10 # For 4 GPU Benchmarks
236+
$ sh run_tf_backend.sh 8_gpu_config lstm_wikitext2 False 10 # For 8 GPU Benchmarks
237237
```
238238

239239

@@ -251,10 +251,10 @@ For MXNet backend benchmarks:
251251

252252
For TensorFlow backend benchmarks:
253253
```
254-
$ sh run_tf_backend.sh cpu_config lstm_synthetic False 20 # For CPU Benchmarks
255-
$ sh run_tf_backend.sh gpu_config lstm_synthetic False 20 # For 1 GPU Benchmarks
256-
$ sh run_tf_backend.sh 4_gpu_config lstm_synthetic False 20 # For 4 GPU Benchmarks
257-
$ sh run_tf_backend.sh 8_gpu_config lstm_synthetic False 20 # For 8 GPU Benchmarks
254+
$ sh run_tf_backend.sh cpu_config lstm_synthetic False 10 # For CPU Benchmarks
255+
$ sh run_tf_backend.sh gpu_config lstm_synthetic False 10 # For 1 GPU Benchmarks
256+
$ sh run_tf_backend.sh 4_gpu_config lstm_synthetic False 10 # For 4 GPU Benchmarks
257+
$ sh run_tf_backend.sh 8_gpu_config lstm_synthetic False 10 # For 8 GPU Benchmarks
258258
```
259259

260260
## References

‎benchmark/benchmark_result/RNN_result.md‎

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -9,16 +9,24 @@
99

1010
Please see [RNN with Keras-MXNet document](../docs/mxnet_backend/using_rnn_with_mxnet_backend.md) for more details on
1111
the poor CPU training performance and unsupported functionalities.
12-
12+
13+
### Configuration
14+
| | |
15+
| :--------------- | :----------------------------------------------------------- |
16+
| Keras | v2.1.6 |
17+
| TensorFlow | v1.8.0 |
18+
| MXNet | v1.2.0 |
19+
| CUDA | v9.0.176 |
20+
| cuDNN | v7.0.1 |
1321

1422
### LSTM-Nietzsche
1523

1624
| Instance Type | GPUs | Batch Size | Keras-MXNet (Time/Epoch), (GPU Mem) | Keras-TensorFlow (Time/Epoch), (GPU Mem) |
1725
|---|---|---|---|---|
1826
| C5.18X Large | 0 | 128 | 78 sec, N/A | 55 sec, N/A|
19-
| P3.8X Large | 1 | 128 | 52 sec, 792 MB | 51 sec, 15360 MB|
20-
| P3.8X Large | 4 | 128 | 47 sec, 770 MB | 87 sec, 15410 MB |
21-
| P3.16X Large | 8 | 128 | TBD | TBD |
27+
| P3.8X Large | 1 | 128 | 52 sec, 792 MB | 83 sec, 15360 MB|
28+
| P3.8X Large | 4 | 128 | 47 sec, 770 MB | 117 sec, 15410 MB |
29+
| P3.16X Large | 8 | 128 | 72 sec, 826 MB | 183sec, 15408TBD |
2230

2331
### LSTM-WikiText2
2432

@@ -27,7 +35,7 @@ Please see [RNN with Keras-MXNet document](../docs/mxnet_backend/using_rnn_with_
2735
| C5.18X Large | 0 | 128 | 1345 sec, N/A | 875, N/A |
2836
| P3.8X Large | 1 | 128 | 868 sec, 772 MB | 817, 15360 MB |
2937
| P3.8X Large | 4 | 128 | 775 sec, 764 MB | 1468, 15410 MB |
30-
| P3.16X Large | 8 | 128 | TBD | TBD |
38+
| P3.16X Large | 8 | 128 | 1214 sec, 826 MB | 3176 sec, 15410 MB |
3139

3240
### Synthetic Data
3341

@@ -36,7 +44,7 @@ Please see [RNN with Keras-MXNet document](../docs/mxnet_backend/using_rnn_with_
3644
| C5.18X Large | 0 | 128 | 24 sec, N/A | 14 sec, N/A|
3745
| P3.8X Large | 1 | 128 | 13 sec, 792 MB | 12 sec, 15360 MB|
3846
| P3.8X Large | 4 | 128 | 12 sec, 770 MB | 21 sec, 15410 MB |
39-
| P3.16X Large | 8 | 128 | TBD | TBD |
47+
| P3.16X Large | 8 | 128 | 19 sec, 826 MB | 49 sec, 15360 MB |
4048

4149

4250
# Detailed RNN Benchmark Results

‎benchmark/scripts/benchmark_resnet.py‎

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -86,8 +86,7 @@
8686

8787
# prepare logging
8888
# file name: backend_data_format_dataset_model_batch_size_gpus.log
89-
log_file = K.backend() + '_' + K.image_data_format() + '_' + args.dataset + '_resnet_v' + args.version + \
90-
'_' + args.layers + '_batch_size' + str(batch_size) + '_' + str(num_gpus) + 'gpus'
89+
log_file = K.backend() + '_' + K.image_data_format() + '_' + args.dataset + '_resnet_v' + args.version + '_' + args.layers + '_batch_size' + str(batch_size) + '_' + str(num_gpus) + 'gpus' # nopep8
9190
logFormatter = logging.Formatter('%(asctime)s [%(threadName)-12.12s] [%(levelname)-5.5s] %(message)s')
9291
rootLogger = logging.getLogger()
9392

@@ -300,9 +299,9 @@ def lr_schedule(epoch):
300299
batch_time = 1000 * (end_time - start_time)
301300
speed = batch_size * 1000.0 / batch_time if batch_time != 0 else 0
302301
rootLogger.info('batch {}/{} loss: {} accuracy: {} '
303-
'time: {}ms speed: {}'.format(int(current_index / batch_size),
304-
int(nice_n / batch_size), loss, accuracy,
305-
batch_time, speed))
302+
'time: {}ms speed: {}'.format(int(current_index / batch_size),
303+
int(nice_n / batch_size), loss, accuracy,
304+
batch_time, speed))
306305

307306
rootLogger.info('finish epoch {}/{} total epoch time: {}ms'.format(i, epochs, total_time))
308307

@@ -323,4 +322,4 @@ def lr_schedule(epoch):
323322
# Score trained model.
324323
scores = model.evaluate(x_test, y_test, verbose=1)
325324
rootLogger.info('Test loss: %.4f' % scores[0])
326-
rootLogger.info('Test accuracy: %.4f'% scores[1])
325+
rootLogger.info('Test accuracy: %.4f' % scores[1])

‎benchmark/scripts/models/lstm_synthetic.py‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -44,9 +44,7 @@ def __init__(self):
4444
def run_benchmark(self, gpus=0, inference=False, use_dataset_tensors=False, epochs=20):
4545
# prepare logging
4646
# file name: backend_data_format_dataset_model_batch_size_gpus.log
47-
log_file = keras.backend.backend() + '_' + keras.backend.image_data_format() + \
48-
'_lstm_synthetic_batch_size_' + \
49-
str(self.batch_size) + '_' + str(gpus) + 'gpus.log'
47+
log_file = keras.backend.backend() + '_' + keras.backend.image_data_format() + '_lstm_synthetic_batch_size_' + str(self.batch_size) + '_' + str(gpus) + 'gpus.log' # nopep8
5048
logging.basicConfig(level=logging.INFO, filename=log_file)
5149

5250
self.epochs = epochs

‎benchmark/scripts/models/lstm_text_generation.py‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -49,9 +49,7 @@ def __init__(self, dataset_name=None):
4949
def run_benchmark(self, gpus=0, inference=False, use_dataset_tensors=False, epochs=20):
5050
# prepare logging
5151
# file name: backend_data_format_dataset_model_batch_size_gpus.log
52-
log_file = keras.backend.backend() + '_' + keras.backend.image_data_format() + \
53-
'_lstm_test_generation_' + self.dataset_name + '_batch_size_' + \
54-
str(self.batch_size) + '_' + str(gpus) + 'gpus.log'
52+
log_file = keras.backend.backend() + '_' + keras.backend.image_data_format() + '_lstm_test_generation_' + self.dataset_name + '_batch_size_' + str(self.batch_size) + '_' + str(gpus) + 'gpus.log' # nopep8
5553
logging.basicConfig(level=logging.INFO, filename=log_file)
5654

5755
self.epochs = epochs

‎benchmark/scripts/models/resnet50_benchmark.py‎

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,6 @@
1717
from keras import backend as K
1818

1919

20-
21-
2220
def crossentropy_from_logits(y_true, y_pred):
2321
return keras.backend.categorical_crossentropy(target=y_true,
2422
output=y_pred,
@@ -39,12 +37,11 @@ def __init__(self):
3937
def run_benchmark(self, gpus=0, inference=False, use_dataset_tensors=False, epochs=20):
4038
self.epochs = epochs
4139
if gpus > 1:
42-
self.batch_size = self.batch_size*gpus
40+
self.batch_size = self.batch_size * gpus
4341

4442
# prepare logging
4543
# file name: backend_data_format_dataset_model_batch_size_gpus.log
46-
log_file = K.backend() + '_' + K.image_data_format() + '_synthetic_resnet50_batch_size_' + \
47-
str(self.batch_size) + '_' + str(gpus) + 'gpus.log'
44+
log_file = K.backend() + '_' + K.image_data_format() + '_synthetic_resnet50_batch_size_' + str(self.batch_size) + '_' + str(gpus) + 'gpus.log' # nopep8
4845
logging.basicConfig(level=logging.INFO, filename=log_file)
4946

5047
print("Running model ", self.test_name)

‎keras/__init__.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,4 +23,4 @@
2323
from .models import Model
2424
from .models import Sequential
2525

26-
__version__ = '2.1.6'
26+
__version__ = '2.1.6'

‎keras/backend/mxnet_backend.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2593,13 +2593,13 @@ def rnn(step_function, inputs, initial_states,
25932593
'Ex: new_x_train = keras.preprocessing.sequence.pad_sequences(old_x_train, '
25942594
'maxlen=MAX_LEN_OF_INPUT_SAMPLE_TYPE_INT). '
25952595
'More Details - '
2596-
'https://github.com/awslabs/keras-apache-mxnet/tree/master/docs/mxnet_backend/using_rnn_with_mxnet_backend.md') #nopep8
2596+
'https://github.com/awslabs/keras-apache-mxnet/tree/master/docs/mxnet_backend/using_rnn_with_mxnet_backend.md') # nopep8
25972597

25982598
if not unroll and dshape[1] is not None:
25992599
warnings.warn('MXNet Backend: `unroll=False` is not supported yet in RNN. Since the input_shape is known, '
26002600
'setting `unroll=True` and continuing the execution.'
26012601
'More Details - '
2602-
'https://github.com/awslabs/keras-apache-mxnet/tree/master/docs/mxnet_backend/using_rnn_with_mxnet_backend.md', stacklevel=2) #nopep8
2602+
'https://github.com/awslabs/keras-apache-mxnet/tree/master/docs/mxnet_backend/using_rnn_with_mxnet_backend.md', stacklevel=2) # nopep8
26032603

26042604
# Split the inputs across time dimension and generate the list of inputs
26052605
# with shape `(samples, ...)` (no time dimension)

‎keras/utils/np_utils.py‎

Lines changed: 20 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -84,34 +84,27 @@ def to_channels_first_helper(np_data):
8484
if data_format not in {'channels_first', 'channels_last'}:
8585
raise ValueError('Unknown data_format ' + str(data_format))
8686

87-
if data_format == 'channels_first':
88-
shape = np_data.shape
89-
if len(shape) == 5:
90-
np_data = np.transpose(np_data, (0, 4, 1, 2, 3))
91-
elif len(shape) == 4:
92-
np_data = np.transpose(np_data, (0, 3, 1, 2))
93-
elif len(shape) == 3:
94-
raise ValueError(
95-
'Your data is either a textual data of shape '
96-
'`(num_sample, step, feature)` or a grey scale image of '
97-
'shape `(num_sample, rows, cols)`. '
98-
'Case 1: If your data is time-series or a textual data'
99-
'(probably you are using Conv1D), then there is no need of '
100-
'channel conversion.'
101-
'Case 2: If your data is image(probably you are using '
102-
'Conv2D), then you need to reshape the tension dimensions '
103-
'as follows:'
104-
'`shape = x_input.shape`'
105-
'`x_input = x_input.reshape(shape[0], 1, shape[1], shape[2])`'
106-
'Note: Do not use `to_channels_fir()` in above cases.')
107-
else:
108-
raise ValueError('Your input dimension tensor is incorrect.')
87+
shape = np_data.shape
88+
if len(shape) == 5:
89+
np_data = np.transpose(np_data, (0, 4, 1, 2, 3))
90+
elif len(shape) == 4:
91+
np_data = np.transpose(np_data, (0, 3, 1, 2))
92+
elif len(shape) == 3:
93+
raise ValueError(
94+
'Your data is either a textual data of shape '
95+
'`(num_sample, step, feature)` or a grey scale image of '
96+
'shape `(num_sample, rows, cols)`. '
97+
'Case 1: If your data is time-series or a textual data'
98+
'(probably you are using Conv1D), then there is no need of '
99+
'channel conversion.'
100+
'Case 2: If your data is image(probably you are using '
101+
'Conv2D), then you need to reshape the tension dimensions '
102+
'as follows:'
103+
'`shape = x_input.shape`'
104+
'`x_input = x_input.reshape(shape[0], 1, shape[1], shape[2])`'
105+
'Note: Do not use `to_channels_fir()` in above cases.')
109106
else:
110-
warnings.warn(
111-
'`to_channels_first()` method transform the data from'
112-
'`channels_last` format to `channels_first` format. Please '
113-
'check the `image_data_format` and `backend` in `keras.json` '
114-
'file.', stacklevel=2)
107+
raise ValueError('Your input dimension tensor is incorrect.')
115108
return np_data
116109

117110
assert data is not None, "A Numpy data should not be None"

0 commit comments

Comments
 (0)