Batch runner doesn't collect data on last step

[PeakEwan]

Describe the bug
When using batch_run the datacollector misses off the last step.

Expected behavior
When using batch_run the datacollector doens't miss of the last step.

To Reproduce

MRE:

import mesa
import pandas as pd

class Foo(mesa.Agent):
    def __init__(self, model: mesa.Model) -> None:
        super().__init__(model)
        self.tally = 0
        
    def step(self) -> None:
        self.tally += 1

class FooHaus(mesa.Model):
    def __init__(self, foos) -> None:
        super().__init__()
        for f in range(foos):
            Foo(self)
        self.datacollector = mesa.DataCollector(agent_reporters={
            "Tally": "tally" #Need something in the datacollector
        })
    
    def check_end_condition(self) -> None:
        if self.steps >= 3:
            self.running = False
            print("Integer of final model step = ", self.steps)
            print("Tally from Foo agent = ", self.agents[0].tally)

    def step(self) -> None:
        self.agents.shuffle_do("step")
        self.datacollector.collect(self)
        self.check_end_condition()

print("1st case: Run model then display agent_vars dataframe")

test = FooHaus(1)
test.run_model()
test_results = test.datacollector.get_agent_vars_dataframe()
print(test_results)

print("------------------------------")
print("2nd case: batch_run then display results from data collector")
batch_run_results = mesa.batch_run(
    model_cls=FooHaus,
    parameters={"foos":1},
    number_processes=1,
    iterations=1,
    data_collection_period=1,
    max_steps=1000,
    display_progress=False
)

batch_run_results_df = pd.DataFrame(batch_run_results)
print(batch_run_results_df)

END MRE

Additional context

The batch_run datacollector also adds a zero step, I'm not sure if this is intended or not but I can see the value of this for viewing an initial state.

I think the issue is in the function _model_run_func in the mes.batchrunner source code when defining the list of steps that _collect_data is run on.

steps = list(range(0, model.steps, data_collection_period))
    if not steps or steps[-1] != model.steps - 1:
        steps.append(model.steps - 1)

 for step in steps:
        model_data, all_agents_data = _collect_data(model, step)

I believe the issue could be that the steps used for model collection goes up to the penultimate step due to model.steps - 1

[satyamlal]

Hi @PeakEwan,
I would like to contribute to resolving this issue and believe I can provide a suitable fix. Could you please assign it to me, or should I proceed directly with the work?

[EwoutH]

@satyamlal go ahead. Also please read:

• Standardized order of events in a step and defining the first step #2229

That’s where we determined that the first step is called step 1 (and not 0).

[quaquel]

Having closed #2588, I looked at the example.

This prints

1st case: Run model then display agent_vars dataframe
Integer of final model step =  3
Tally from Foo agent =  3
              Tally
Step AgentID       
1    1            1
2    1            2
3    1            3
------------------------------
2nd case: batch_run then display results from data collector
Integer of final model step =  3
Tally from Foo agent =  3
   RunId  iteration  Step  foos  AgentID  Tally
0      0          0     0     1      NaN    NaN
1      0          0     1     1      1.0    1.0
2      0          0     2     1      1.0    2.0

So, rather than not gathering the last step, the issue is that the step counter shown by batch_run uses zero-based indexing instead of 1-based indexing. The proper fix is thus to change batch_run to use the steps as counted by the model.