Add secondary custom callback for logging step wise information

[danielwolff1]

With this PR, I propose the implementation of a second callback class, similar to the already existing EpisodeLogCallback, which tracks stepwise information.

The motivation for this extension was to simplify the evaluation of trainings by allowing to better understand the policy learned by the agent from investigating and reconstructing the trajectory (i.e., sequence of actions) undertaken in each episode.

Overview of the StepInformationLogCallback

Information logged

The following information is currently collected in each step and will be logged to a separate csv file:

• current episode
• current step
• undertaken action
• observation
• reward

Configuration possibilities

The StepInformationLogCallback will create a csv file similar to the already existing epsiode_log.csv which contains the information outlined above. At the moment, the frequency with which information will be written to the csv file can be configured via the step_log_update parameter in the input .json file:

• the default value is episode, meaning that the callback will record the information from all steps within an episode and only write new information to the csv file once the episode is completed
• alternatively an arbitrary integer can be provided which will then trigger output every n steps

Open questions / issues

General questions

• I'm not sure if we in fact need the reward as we already get the accumulated/discounted reward from the EpisodeLogCallback
• I'm open to change the name of the class, as I only noticed now while creating the PR that the name StepInformationLogCallback differs in style from the already existing EpisodeLogCallback

Problems during testing

• On my system (Mac OS Sonoma 14.6.1) I couldn't successfully run the tests as I got the error ERROR tests/test_util_funcs.py - AttributeError: module 'numpy' has no attribute 'float128' when trying to invoke pytest in the top level directory of the repository from within a freshly created conda environment. However I tested the callback in a small example script and will provide the output blow as additional information to this PR.

Additional information

An example step_log.csv from a simple test of this callback can be found below:
step_log.csv

[clemens-fricke]

Looks very good. Thanks for the work. I have a few suggestions to streamline and reduce doubled variables.

There might be some comments that are not 100% correct if you use the suggestions directly. Also, as always, changes via github are not tested :D.

[clemens-fricke]

The test error is most likely due to different numpy version. I think I had changed that exact test during the last PR. Let's look into that so that this error is not persistent.

[clemens-fricke]

Thanks for the really good work.