====== Software Setup ======

  - Open the Visual Studio Code from the Applications directory on the flash drive
  - Click on the small green square in the bottom left corner of the screen\\ \\ <WRAP box>{{ :pibot:lesson5:softwaremac:s1.png |VSCode Main Screen}}</WRAP>
  - Select the "Connect Current Window to Host..." option\\ \\ <WRAP box>{{ :pibot:lesson5:softwaremac:s2.png |VSCode Remote-SSH}}</WRAP>
  - Type "pi@pibot##.local" and press enter\\ \\ <WRAP round important>Replace the text "pibot##" with the label on the pibot</WRAP> <WRAP round important>The password to enter when prompted is "DogsAndCatsAreNice2." without the quotes. The '.' at the end is important!</WRAP> <WRAP round tip>It may take a couple of minutes while it installs some files on the PiBot</WRAP> <WRAP box>{{ :pibot:lesson5:softwaremac:s3.png |VSCode Remote Connection}}</WRAP>
  - Click the "Open Folder" button\\ \\ <WRAP box>{{ :pibot:lesson5:softwaremac:s4.png |VSCode Open Folder}}</WRAP>
  - Select the "pi" folder and click the "OK" button\\ \\ <WRAP box>{{ :pibot:lesson5:softwaremac:s5.png |VSCode Browse}}</WRAP>
  - Open a terminal in Visual Studio Code from the Terminal->New Terminal menu
  - Type the following commands and press enter after each line to install the software required for this lesson\\ \\ <code>sudo apt-get update
sudo apt-get install python3-picamera python3-pil python3-opencv libjpeg-dev libfreetype6 libfreetype6-dev zlib1g-dev libwebp-dev libopenjp2-7-dev libatlas-base-dev fonts-freefont-ttf git</code>
  - Type the following commands and press enter after each line to install the Python libraries required for this lesson\\ \\ <code>sudo pip3 install --upgrade numpy
sudo pip3 install imutils</code>
  - Type the following command and press enter to install the Python Tensorflow Lite runtime for this lesson\\ \\<code>sudo pip3 install https://github.com/google-coral/pycoral/releases/download/v2.0.0/tflite_runtime-2.5.0.post1-cp37-cp37m-linux_armv7l.whl</code>
  - Type the following command and press enter to download the Python code for this lesson\\ \\ <code>git clone https://git.jonathan.struebel-family.net/robot-club/pibot/lesson5.git</code>
  - Type the following command and press enter to change into the lesson 5 directory\\ \\ <code>cd lesson5</code>
  - Type the following commands and press enter after each line to download the object detection models\\ \\ <code>curl -L 'https://tfhub.dev/tensorflow/lite-model/efficientdet/lite0/detection/metadata/1?lite-format=tflite' -o efficientdet_lite0.tflite
curl -L 'https://storage.googleapis.com/download.tensorflow.org/models/tflite/edgetpu/efficientdet_lite0_edgetpu_metadata.tflite' -o efficientdet_lite0_edgetpu.tflite</code>
  - Type the following command and press enter to start the object detection MJPEG Streamer server for this lesson\\ \\ <code>python3 webstreaming.py</code>
  - Open the Safari App, type http://pibot##.local:8080 in the address bar, and press enter to test the object detection MJPEG Streamer software\\ \\ <WRAP round important>Replace the text "pibot##" with the label on the pibot</WRAP> <WRAP round important>Verify that the PiBot detects objects in front of the camera by drawing a box around them</WRAP> <WRAP box>{{ :pibot:lesson5:softwaremac:s6.png |Object Detection MJPEG streamer Test}}</WRAP>
  - Select the "osoyoowebcar" directory on the left and then click on the templates directory and select the index.html file\\ \\ <WRAP box>{{ :pibot:lesson5:softwaremac:s7.png |VSCode index.html}}</WRAP>
  - Replace the **8899** on line 34 with **8080**
  - Save the change by using the File->Save menu
  - Open a terminal in Visual Studio Code from the Terminal->New Terminal menu
  - Type the following command and press enter to run the Python code for this lesson\\ \\ <code>cd osoyoowebcar
sudo python3 webcar.py</code>
  - Open the Safari App, type http://pibot##.local in the address bar, and press enter\\ \\ <WRAP round important>Replace the text "pibot##" with the label on the pibot</WRAP> <WRAP box>{{ :pibot:lesson5:softwaremac:s8.png |Webcar app}}</WRAP>
  - Drive the PiBot around and see what objects it detects

====== Software Explanation ======

The webstreaming.py file is another Flask app that streams images from the CSI Camera connected to the Raspberry Pi to your web browser after searching for objects within the image.

  - In the webstreaming.py file, find the decorators. What do you think each of the functions is doing?\\ \\ <WRAP round tip>If you need a refresher on what decorators are, review [[:pibot:lesson4:softwaremac#software_explanation|Lesson 4]]</WRAP>
  - Find the generate function. What do you think this function does? Where is the function called?\\ \\ <WRAP round tip>outputFrame is a global buffer that stores exactly one image from the camera. The cv2.imencode function converts (encodes) an image frame into a standard format, in this case the JPEG format</WRAP> <WRAP round tip>The yield function temporarily stops the loop that it's in and returns the data that is passed as a parameter to the yield function.</WRAP>
  - Find the location in the code where data is saved to the outputFrame variable. What function is it in? What do you think this function does?\\ \\ <WRAP round tip>The while loop reads each image from the camera, performs object detection on it, draws boxes around the objects, and prints some statistics on the image before saving the image to the global outputFrame buffer</WRAP> <WRAP round tip>The detector variable is an instance of the ObjectDetector class. It get initialized on line 127</WRAP> <WRAP round tip>The detect_objects function is run as a separate thread. Look at line 221 to see how to do that. Threads are a powerful way to allow different functions in the code to run in parallel. Using threads can help your code run faster.</WRAP>
  - Open the object_detector.py file\\ \\ <WRAP box>{{ :pibot:lesson5:softwaremac:s9.png |object_detector.py}}</WRAP>
  - Find the detect function in the ObjectDetector class. Explore the different functions to see what it is doing.\\ \\ <WRAP round tip>When self is used in front of a function call it is referring to a function defined within that same class</WRAP> <WRAP round tip>The detect function uses the Tensorflow Lite (or TFLite) library to run machine learning neural networks that process data and provide information about what is in that data. Tensorflow Lite was developed by Google. Learn more about machine learning and neural nets in [[:pibot:lesson5#algorithm_explanation|Algorithm Explanation]].</WRAP>
  - When you're finished driving the PiBot around, hold down the control + C keys and type the following command in the Visual Studio Code terminal to shutdown the PiBot\\ \\ <code>sudo shutdown -h now</code>
  - Click on the small green rectangle in the bottom left corner of the screen and select "Close Remote Connection"
  - Wait 3-5 minutes for the green light on the Raspberry Pi to stop flashing and turn off the switch on the PiBot battery

[[:pibot:lesson5#software|Return to Lesson 5]]