readme format fix

Author: zenglhardt

readme.md

@@ -18,7 +18,6 @@ The CLI will interface with the same system configuration info as the Arduino ID
 ## Running Your First Test
 To run your first test, first edit the ``llm_prompts.json`` file to include the prompt you want to test. The format of this file is as follows, note that you can choose to specify any number of prompts in the file, which could be useful if you want to compare the performance of different prompting strategies for a task. The example below includes two different prompt examples:
 ```
-{
 {
     "llm_prompts": [
         {
@@ -38,17 +37,17 @@ Next, open the ``testbench.py`` file and adjust the script parameters at the top
 -__temperature__: the temperature to use for the LLM. 
 - __repetition__: the number of times to try each prompt
 - __max_calls_per_minute__: the maximum number of calls to the LLM API to make per minute. This is useful to avoid hitting the API rate limit for models that have a limit expressed in requests per minute. Most APIs have now moved on to tokens/min for limits. 
-- __port__: the port on the computer that the Arduino to run LLM code is connected to. You can find this port by opening Arduino IDE and seeing which port the different boards are connected to. It might look something like ``COM3`` or ``/dev/cu.usbmodem14101`` depending on your operating system. 
--__servo_port__: the port on the computer that is controlling the servo, currently used for IMU tests
--__verify_port__: the port on the computer that the Arduino to run human-written code is connected to. This is used to verify the output of the LLM code.
--__arduino_type__: the type of Arduino connected to ``port``. This is used to determine the correct board type to use when calling the build and upload commands in the Arduino CLI
--__scratch_name__: the name of the scratch file to use to store the LLM generated code before it is compiled and uploaded to the Arduino. You should only change this if for some reason Arduino CLI is having issues accessing the file in the project directory. 
--__save_programs__: whether or not to save each generated program to the ``programs`` directory. This is useful for debugging and for generating a dataset of programs for future use.
--__baudrate__: the baudrate to use for the serial connection to the Arduino. This should match the baudrate specified in your LLM prompts as well!
--__mode__: the type of test to run. Currently supports 3 tests - ``check_string``, ``1D``, and ``IMU``.
--__timeout__: the timeout for how long to listen for serial output from the connected Arduino boards. For most tests, something between 5-10s should be sufficient. 
--__expected_msg__: for the ``check_string`` test, this is the string to expect from the Arduino running human-written code. 
--__threshold__: for the ``1D`` test, this is the threshold for the maximum euclidean distabce between the LLM and human-written code outputs. For the ``IMU`` test, this is the threshold for the allowable difference in average values for each position and each IMU axis. To find the appropriate threshold, test with human-written code first to understand the expected range of values for correct code. 
+- __port__: the port on the computer that the Arduino to run LLM code is connected to. You can find this port by opening Arduino IDE and seeing which port the different boards are connected to. It might look something like ``COM3`` or ``/dev/cu.usbmodem14101`` depending on your operating system.
+- __servo_port__: the port on the computer that is controlling the servo, currently used for IMU tests
+- __verify_port__: the port on the computer that the Arduino to run human-written code is connected to. This is used to verify the output of the LLM code.
+- __arduino_type__: the type of Arduino connected to ``port``. This is used to determine the correct board type to use when calling the build and upload commands in the Arduino CLI
+- __scratch_name__: the name of the scratch file to use to store the LLM generated code before it is compiled and uploaded to the Arduino. You should only change this if for some reason Arduino CLI is having issues accessing the file in the project directory. 
+- __save_programs__: whether or not to save each generated program to the ``programs`` directory. This is useful for debugging and for generating a dataset of programs for future use.
+- __baudrate__: the baudrate to use for the serial connection to the Arduino. This should match the baudrate specified in your LLM prompts as well!
+- __mode__: the type of test to run. Currently supports 3 tests - ``check_string``, ``1D``, and ``IMU``.
+- __timeout__: the timeout for how long to listen for serial output from the connected Arduino boards. For most tests, something between 5-10s should be sufficient. 
+- __expected_msg__: for the ``check_string`` test, this is the string to expect from the Arduino running human-written code. 
+- __threshold__: for the ``1D`` test, this is the threshold for the maximum euclidean distabce between the LLM and human-written code outputs. For the ``IMU`` test, this is the threshold for the allowable difference in average values for each position and each IMU axis. To find the appropriate threshold, test with human-written code first to understand the expected range of values for correct code. 
 
 ### API keys:
 You will need either a Google or OpenAI API key to run this code. You can place the key for each vendor in a file called ``OpenAI.key`` or ``PaLM.key`` in the root directory of the project. The code will automatically read the key from the file and use it to authenticate with the API.

testbench.py

@@ -4,7 +4,6 @@
 
 # Include your PaLM API key in a file called PaLM.key in the same directory as this file.
 # Include your OpenAI API key in a file called OpenAI.key in the same directory as this file.
-import random
 # Select Test Params:
 service = 'OpenAI' #Select either Google or OpenAI or Text (for a folder of program files)
 #model = 'models/text-bison-001' #Model to use for Google/OpenAI