MobilePose-Pi

This is a deployment of MobilePose-pytorch for Raspberry Pi.

Structure

Files	Explanation
cocoapi	this is from https://github.com/cocodataset/cocoapi
mobilepose-pi	execution directory for Raspberry Pi
models	pretrained models (.t7) are stored
pose_dataset	MPII dataset (csv files)
results	keypoints of each model's grountruth & prediction
OpenCV_Installation_Guide.md	OpenCV installation guide
PyTorch_Installation_Guide.md	PyTorch installation guide
README.md	general explanation about this repo
ShuffleNetV2.py	network architecture of ShufflenetV2
Study_on_results.md	study on validation of our results
coco_utils.py	generating groundtruth&prediction json files
dataloader.py	multi-thread dataloader with augmentations
dataset_factory.py	get dataset's keypoint from csv files
estimator.py	this is for run_webcam.py
eval_pc.py	scripts to evaluate each model
mobilenetv2.py	network architecture of MobilenetV2
networks.py	get model's path and input's height & width network architecture of Resnet
pycocotools	link to cocoapi/PythonAPI/pycocotools
requirements.txt	libraries needed to run scripts
run_webcam.py	real-time pose estimation using webcam
training.py	training models

Note: MPII is used for training and COCO is used for evaluation.

Results

Note:As you can see, each model has room to be improved. Any ideas or opinions are welcome!

Network	FPS	Size(MB)	mAP
1.0 ShufflenetV2	1.09	5.3	0.000
1.0 MobilenetV2	0.66	9.3	0.045
Resnet18	0.39	44.9	0.257

You can get more detailed info for this results here.

Requirements

For 'mobilepose-pi', you need following environment.

Raspberry Pi (We used Pi 2 Model B for this repo)
Rasbian stretch (32bit)
Python 2.7

Installation

Before installation, please change swap size.

At Pi terminal, open 'dphys-swapfile'.

sudo nano /etc/dphys-swapfile

and change CONF_SWAPSIZE. (default is set to 100.)

CONF_SWAPSIZE=2048

Activate it.

sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile start

Now you can install following things into your Pi.

PyTorch-0.2.0(499MB) - Follow my guide.
torchvision-0.2.1(2.7MB) - Please install it from source. Repo is here.
OpenCV-3.4.0(1.8GB) - Follow this guide.

Also, don't forget to install libraries.

pip install -r requirements.txt

At the end, change CONF_SWAPSIZE to 100 again.

Execution

You can evaluate your network on PC first.
If the result seems to be good, you can try it on Pi using 'mobilepose-pi'.

For mobilenet:

python eval_pc.py --model mobilenet

For resnet:

python eval_pc.py --model resnet

For shufflenet:

python eval_pc.py --model shufflenet

*I just reused the same pretrained models MobilePose-pytorch author provided.
For shufflenet I used my original model.
Of course, you can use your own models also.

You can try real-time estimation using 'run_webcam.py'
(I didn't change this file from original repo.)
Please connect webcam to PC first. Then try this.

python run_webcam.py --model [name of model]

Currently, run_webcam.py doesn't support shufflenet.

Training

You can train three models (shufflenet/mobilenet/resnet) on your PC.

You need to download MPII training Images(12.9GB). This is from here.
After extraction, please set ROOT_DIR at dataloader.py (line.197).

Different from MobilePose-pytorch(original repo), the command is
python train.py --model=[name_of_model_you_want_to_train] --retrain=[bool]

If you want to train shufflenet, you can do it just change model name.
(i.e. python train.py --model=shufflenet)

Model conversion

Note: development of this feature is currently suspended.

For conversion, I mainly adopted this tutorial.
I recommend to do this via conda.

1.You need to set up PC environment.

Caffe2
ONNX
PyTorch

Citation

Original Repo:
https://github.com/YuliangXiu/MobilePose-pytorch

ShufflenetV2 paper:
https://arxiv.org/pdf/1807.11164.pdf

ShufflenetV2 implementation:
https://github.com/ericsun99/Shufflenet-v2-Pytorch

ONNX:
https://github.com/onnx/onnx

PyTorch installation:
https://wormtooth.com/20180617-pytorch-on-raspberrypi/

torchvision installation:
https://github.com/pytorch/vision

OpenCV installation:
https://www.pyimagesearch.com/2017/09/04/raspbian-stretch-install-opencv-3-python-on-your-raspberry-pi/