-
Notifications
You must be signed in to change notification settings - Fork 8
Car System Installation Notes
The car system depends on independent component: Ubuntu, TX1/Jetpack, stereolab’s zed_ros_wrapper, Caffe, pytorch, opencv2 and ROS.
You need this to connect to the mini-HDMI port for display of the TX1 (the port is on top, perhaps obscured by electrical tape and cables) so that you can use the Ubuntu GUI with a monitor in order to set your network preferences. If you are adept at Ubuntu Linux, you can also do this ‘blind’ by plugging a keyboard into a USB port and typing the necessary configuration commands. This is pretty simple. Power up the TX1 by plugging in the battery. Wait a few minutes to insure it is up. Then type Ctrl-Alt-F1 which will put you into a login terminal outside the GUI. Type in ubuntu as username, the type in ubuntu as password. Wait a moment, then you should be logged in and on a command line. Now type:
nmcli d wifi connect “--your wifi network name here--” password “--your password here--”
This should turn on the wifi connection to your network. Perhaps type slowly . . .
Then, from a separate computer on this same network, you can ssh in. However, you need to find the ip address of the TX1 first. I do this by using my smartphone as a mobile wifi hotspot. It lists all ip addresses of all the devices connected to it by wifi. I will use the numbers of my hotspot for the example below:
ssh ubuntu@192.168.43.63
Password: ubuntu.
It helps to put in a public-key system of course. Now you don’t need the hdmi cable anymore, you have complete control. Note that sudo is set to a no-password state.
In the meantime, if you have your TX1, let’s start setting that up. I’m having difficulty making a copy of the necessary folder (it doesn’t want to zip), but in the worst case you can install by hand, perhaps not a bad idea because then you will know what is involved. Then make your own clone for replication.
You might ask, why not install the TX1 development board directly in the car? The answer was, “It doesn’t fit in the roll cage and we depend on the roll cage for protection.” But with the introduction of the tough outer car bodies this requirement is relaxed and it finally dawned on me to try installing the TX1 dev board as it is. It doesn’t fit, of course, but with a little bit of cutting of the roll cage (which I repair with bamboo and tie-wraps) I can get it in there at a funny angle. It seems to work well, with very stable video. The penalty is that the car has a noticeably higher center of gravity which causes trouble when turning on hills. The robustness of this system to extreme stress has not been tested, though the usual rollovers seem to cause no harm.
I have also modified the ROS files so that the only Arduino necessary is the motor/servo arduino. That means the IMU and GPS/LED arduinos can be left out, greatly simplifying construction. I am going to branch my car design into a very small number of super research cars with all sensors, and a large number of camera-only cars which need to be simple to build.
Once you have access to the TX1, we can go over some basic updates. I have a car ‘Mr. Yellow’ in an identical state as Mr. White, so we can upgrade these in parallel and make sure we are on the same page.
The new network (z2_color.caffemodel) is much more impressive than the original (z2.caffemodel), though it is still an ultra simple model. So I think there are very cool things to be down with even the most simplified version of these cars.
When starting ROS (rla), pay close attention to the text scrolling by. I keep one finger on the CONTROL key and watch out for any red text, at which point I type ‘C’. For example, if you fail to put in the flash drive, you will get an error, but because ros is a multi-node system, an error does not immediately cause it to halt. When everything is set up properly, the ROS system is very reliable.
To start the TX1, plug in TX1 battery. Sometimes the TX1 will not start if the J100 is the carrier. This is an issue with the carrier board. Simply unplug the battery, wait ten seconds, and try again. (It is handy to turn the car upside down when dealing with the batteries. It is handy to give the TX1 plug a distinctive marking because it is easy to confuse the two yellow connectors).
Log in via ssh. The username is ubuntu and the password is ubuntu.
I use three letter aliases to do stuff quickly in the field. rrm clears the rosbag folder, which can accumulate .bag files. rlog clears the ros log, which can grow very large. I type both these commands when logging in. rcd will take you to the rosbag flash drive, a good way to test that it is connected. rla starts ros launch. The LED panel should read 2 (red color). The aliases for these three letter commands are in ~/kzpy3/teg2/bashrc, which is sourced by .bashrc .
Before plugging in the motor battery, turn on the controller and click the up button more than 4+ times to set it to state 1. Then click the down button once, to set it to state 2. Under no circumstances plug in the motor battery when the controller is in state 4, or the car can occasionally drive out of control. Note, before, before you plug in the motor battery, you cannot change the state on the car, although you can change the state of the controller itself. For safety, you can put the car on a box so that the wheels are off the ground (or upside down). However, if the controller is set properly as described above, there will be no problem and the car can be started on the ground.
Now you can click the buttons on the right side to change state.
Control mode states
- human control, data mode (speed limit set in Arduino code)
- human control, intended as non-data mode (no speed limit)
- caffe mode. There is a delay (currently 2s) before caffe is enabled. This allows for passing through state 3 without having an effect.
- calibration mode. It is assumed that the car is still when entering this mode. If this mode is entered with the car moving, the car will continue to move and it is necessary to leave the state to regain control. For example, if in state 3 and the car is moving, first go to state 2 before going to state 4. In state 4, set the steering and motor controls to max and min values and hold briefly at each (0.5 s). This will set the control ranges correctly. This must be done at the beginning of the session or else the car will not function properly. It is good to calibrate periodically. If remaining in state for 30s, the TX1 will shut down as a convenience feature. In summary, enter state 4 when necessary and with caution, and leave it ASAP.
If car appears out of control, press the up button rapidly to enter state 1 or 2.
If you are actually going to go out with the car, use the ‘screen’ command before running rla. Then when rla is running, hit CTRL-A-D to detach. Now even if your laptop is turned off, it will not stop the running of the ros system.
A reason to plug in motor battery after starting with rla is to prevent the car from making an occasional lurch at launch time. This little lurch will disrupt the gyroscope calibration. After starting rla, in another terminal I do rgy which shows the gyroscope data. If these are not zero, quit the ros run and start again.
rhz displays frame acquisition rate. It should be near 30 hz.
Even if I am doing data collection in state 1, I like to do a little driving in mode 3 under full caffe control. This is a good test that everything is working in the motor-steering-camera systems.
If in mode 3 and the car seems unresponsive, something is wrong. If the car is in caffe mode and you pick it up and rotate it, the steering and motor should go wild. If not, it is not working properly. Go to mode 2 to stop, then to mode 4 to recalibrate. There is a bug that can mess things up when applying human driving commands in state 3. Sometimes it causes no problem. All this will be sorted out soon. (My project for the holidays, along with the radar…)
To do: Improve electrical shielding of motor and flash drive Add potentiometer to steering to give ground truth independent of radio control signals
Note: Saving data is sensitive to the brand of usb3 hub, the flash drive, and the drive format (Ext4)
Note: The batteries must not go below 9V or they will be ruined. The car has an automatic cutoff, but the TX1 does not. The TX1 battery should run for 1.5 to 2 hours, but a battery voltage checker is the best measure. I should have included one.
Note: TX1 time is set via the ssh connection. In some circumstances (i.e., wifi hotspot out of range of a tower in the country) the data is not set. In this case, use ssh_date alias, which will get the date from the laptop host. You will need to adjust the actual command details. It is very important to have good date information because of the way ROS uses timestamps for analysis. For example, if you are in the country and the TX1 goes to the default date of 2012, and then you get within range of a cell tower and the date gets set correctly, ROS will consider the data file to have been collected over the course of four years and allocate (or fail to allocate) memory accordingly.
Note: The file ~/kzpy3/teg2/notes/setting_up_TX1.txt holds useful information about setting up the Jetson for Caffe and ROS. Caffe is a moving target. I have installed it many times, but with the latest Jetpack version I am having difficulty. If you figure it out, let me know. I'm sure I'll get it in a day or so. (this is solved, will add info later today)
sudo apt-get --assume-yes install git git clone https://github.com/karlzipser/kzpy3.0.git mv kzpy3.0 kzpy3
vi kzpy3/teg1/setting_up_TX1.txt
sudo apt-get --assume-yes install
libprotobuf-dev protobuf-compiler gfortran
libboost-dev cmake libleveldb-dev libsnappy-dev
libboost-thread-dev libboost-system-dev
libatlas-base-dev libhdf5-serial-dev libgflags-dev
libgoogle-glog-dev liblmdb-dev gcc-4.7 g++-4.7
python-numpy
libboost-filesystem-dev
python-dev
libboost-python-dev
git clone https://github.com/BVLC/caffe.git cd caffe cp Makefile.config.example Makefile.config
make -j 4 all
build/tools/caffe time --model=models/bvlc_alexnet/deploy.prototxt --gpu=0
TX1 Alexnet benchmarks
Batchsize = 10
with GPU
I1007 13:24:02.158314 6710 caffe.cpp:377] Average Forward pass: 171.171 ms.
I1007 13:24:02.158380 6710 caffe.cpp:379] Average Backward pass: 152.4 ms.
I1007 13:24:02.158437 6710 caffe.cpp:381] Average Forward-Backward: 323.99 ms.
sudo apt-get --assume-yes install python-pip sudo easy_install cython
sudo pip install -U scikit-image sudo pip install protobuf
make pycaffe
export PYTHONPATH=~/caffe/python:$PYTHONPATH
############# install ROS ################
sudo update-locale LANG=C LANGUAGE=C LC_ALL=C LC_MESSAGES=POSIX sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu trusty main" > /etc/apt/sources.list.d/ros-latest.list' sudo apt-key adv --keyserver hkp://ha.pool.sks-keyservers.net --recv-key 0xB01FA116 sudo apt-get update sudo apt-get install ros-indigo-desktop
sudo apt-get install python-rosdep sudo rosdep init rosdep update
echo "source /opt/ros/indigo/setup.bash" >> ~/.bashrc source ~/.bashrc
###################
ZED setup: note, ZED_SDK_Linux_JTX1_v1.1.0_32b_JP21.run works, v1.1.1_64b doesn't.
test ZED with: /usr/local/zed/tools/ZED\ Explorer
#################
#################
Add to .bashrc: source /opt/ros/indigo/setup.bash
$ mkdir -p ~/catkin_ws/src $ cd ~/catkin_ws/src $ catkin_init_workspace $ cd ~/catkin_ws/ $ catkin_make
Add to .bashrc: source ~/catkin_ws/devel/setup.bash
#####################
#####################
$ sudo apt-get install libpcl-1.7-all ros-indigo-pcl-ros ros-indigo-image-view $ cd ~/catkin_ws/src $ git clone https://github.com/stereolabs/zed-ros-wrapper.git
It may be necessary to do this: in zed-ros-wrapper/CMakeLists.txt put in: SET(CUDA_VERSION "7.0")
$ cd ~/catkin_ws $ catkin_make
Test that it works $ roslaunch zed_wrapper zed.launch $ rosrun image_view image_view image:=/camera/right/image_rect_color
######################
######################
$ git clone https://github.com/rctn/bair_car.git $ roslaunch bair_car bair_car.launch use_zed:=true $ rosrun image_view image_view image:=/bair_car/zed/right/image_rect_color
And rostopic echo all the topics to check
###########################
###########################
Go into bair_car.launch and change the bagpath to where you want to record data to (e.g. flash drive)
$ roslaunch bair_car bair_car.launch use_zed:=true record:=true
######### prompt,e.g. ######### PS1='Mr_Audi:\W\ $ '
############## screen ########
sudo apt-get install screen
To disconnect (but leave the session running), Hit Ctrl + A and then Ctrl + D in immediate succession. You will see the message [detached]
##############
####### sudo, no password #########
sudo visudo
ALL=(ALL) NOPASSWD: ALL
########### SSH login without password, see: ############
mplayer -fps 26 mf://*.png
git config --global credential.helper "cache --timeout=3600"
################# setting time from command line (for when in Tilden) ##############
Set Time To change time means to set a new time. To set time in Ubuntu (or any Linux), just run the following command sudo date newdatetimestring where newdatetimestring has to follow the format nnddhhmmyyyy.ss which is described below nn is a two digit month, between 01 to 12 dd is a two digit day, between 01 and 31, with the regular rules for days according to month and year applying hh is two digit hour, using the 24-hour period so it is between 00 and 23 mm is two digit minute, between 00 and 59 yyyy is the year; it can be two digit or four digit: your choice. I prefer to use four digit years whenever I can for better clarity and less confusion ss is two digit seconds. Notice the period ‘.’ before the ss. Let’s say you want to set your computer’s new time to December 6, 2007, 22:43:55, then you would use: sudo date 120622432007.55 ########
############# data collection shortcuts (smartphone login) #######
alias rhz='rostopic hz /bair_car/zed/left/image_rect_color' alias rls='ls -al /media/ubuntu/rosbags' alias rrm='rm catkin_ws/src/bair_car/rosbags/' alias rlog='rm ~/.ros/log/' alias rla='roslaunch bair_car bair_car.launch use_zed:=true record:=true' alias rlac='roslaunch bair_car bair_car.launch use_zed:=true record:=true caffe:=true' alias rgy='rostopic echo /bair_car/gyro' alias rgp='rostopic echo /bair_car/gps' alias rac='rostopic echo /bair_car/acc' alias rst='rostopic echo /bair_car/steer' alias rmo='rostopic echo /bair_car/motor' alias ren='rostopic echo /bair_car/encoder' alias rcd='cd /media/ubuntu/rosbags' alias ssd='sudo shutdown -h now' alias rcn='echo $COMPUTER_NAME' alias riv='rosrun image_view image_view image:=/bair_car/zed/right/image_rect_color'
alias test_caffe='cd ~/caffe; build/tools/caffe time --model=models/bvlc_alexnet/deploy.prototxt --gpu=0' alias zede=
#ln -s ~/kzpy3/teg2/bdd_car_versions/bdd_car_GPS2/bair_car/ bair_car ######################## for .bashrc from MacBook #################
#echo "source /8August2016_common_aliases"
#export DISPLAY=:0.0
alias gacp="git add -u;git commit -m 'gacp';git push origin master"
alias gckzpy3="git clone https://github.com/karlzipser/kzpy3.0.git"
alias ipy="ipython --no-banner"
export PYTHONPATH=:$PYTHONPATH
#export PYTHONPATH=/kzpy3/caf/layers:$PYTHONPATH
#export PYTHONPATH=/kzpy3/caf2/layers:$PYTHONPATH
export PYTHONSTARTUP=/kzpy3/vis.py
export PATH=/kzpy3/scripts:$PATH
export COMPUTER_NAME="Mr. ---"
######################################################
git clone repoman@repo.eecs.berkeley.edu:projects/bdd/rc-car.git
http://unix.stackexchange.com/questions/22367/where-are-networkmanagers-wifi-settings-stored ls /etc/NetworkManager/system-connections/
set date over network:
sudo date --set="$(ssh karlzipser@192.168.43.243 date)"
http://wiki.ros.org/web_video_server
PS1="Mr_Silver: \W$ " #"Mr_Blue_original_clone: \W$ " export PATH=/usr/local/cuda-7.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin export LD_LIBRARY_PATH=/usr/local/cuda-7.0/lib: export __GL_PERFMON_MODE=1
source /opt/ros/indigo/setup.bash source ~/catkin_ws/devel/setup.bash source ~/kzpy3/teg2/bashrc
if [ -n "$SSH_CLIENT" ] || [ -n "$SSH_TTY" ]; then SESSION_TYPE=remote/ssh echo "This is a remote/ssh login" echo "sleep before rstat" sleep 30 rstat
else echo "sleep before rla..." sleep 30 rrm rlog rla
fi
sudo service network-manager restart
sudo nmcli d wifi connect "AM_phone" password "" sudo rm -r kzpy3 git clone https://github.com/karlzipser/kzpy3.1.git mv kzpy3.1 kzpy3 git clone https://github.com/karlzipser/caffe_current.git
.bashrc auto login wifi ethernet model repo code repo
3 July 2017 power up ESC delete and reenter network connections bind receiver (keep transmitter very close, press in button. if light flickers, turn transmitter odd and on) flash Arduino code do git pull currently need to reboot every time running rla because of ZED issue
On kzpy3 master, chmod a+x teg2/bdd_car_versions/bdd_car_rewrite_SD1/bair_car/nodes/get_ros_images_live2_z2_color_TX1.py if necessary.
9 July 2017