-
Notifications
You must be signed in to change notification settings - Fork 942
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adds Altair plot component and some other fixes #2644
base: main
Are you sure you want to change the base?
Conversation
Performance benchmarks:
|
Thanks for the PR! A few of the automated checks broke, could you look into what caused that? I will leave the review to others. |
@nissu99 also please look at #2643 and #2641 As you are @sanika-n are working in the same space As it looks like you two are looking over the entire altair implementation I would also recommend looking at #2642 discussion It is always good to collaborate and think together you may address some larger visualization challenges. |
"alt defined"
for more information, see https://pre-commit.ci
@tpike3 When I run the tests locally, they don't fail, but they are failing here. |
I am not sure why your tests are passing locally, however for tests to pass they need to updated. It is expecting
|
"removed a line"
"updated toml"
"removed suppress"
for more information, see https://pre-commit.ci
"doc string "
for more information, see https://pre-commit.ci
"operator bug"
Thank You, the issue got resolved by adding altair to the toml file. |
Removing stuff that has become redundant is fine. |
@quaquel i have added all the stuff,can you review it? |
I'll try to review it over the weekend, but I am rather bussy at the moment. This is a part of the code base I am not intimately familiar with and his been a while since I used Altair, so reviewing will take more time than parts of the code base I know inside out. |
Really sorry for the delay ,was travelling due to some family reasons,I am working on changes to make the api similar to that of matplotlib. |
@nissu99 can you give an indication of your timeline? We might otherwise merge some other altair PR first. |
As you had pointed out, I was seeing if a common overarching function could be made for both Matplotlib and Altair stuff. |
@quaquel what do you say should I focus on that or just make other changes and will make a future PR for the Common function? |
This PR is already an improvement. The logical next step, which ideally would be part of this PR, is to structure the code exactly as done for matplotlib. This, for example, means having a single overarching method to get agent data and updating agent positions for e.g., networks, and hexgrids inside their respective draw method. The last step, and well beyond this PR is to see what we can harmonize across altair and matplotib. |
made the changes , is this fine? |
@nissu99, is the grid drawing logic implemented yet, because I am unable to see the grid lines? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@nissu99 I have done a basic overview of functioning of the altair
drawing system, if you find similar problems please fix them. I will do a thorough review on a later date.
return _draw_legacy_grid(space, all_agent_data, agent_portrayal) | ||
case HexGrid(): | ||
return _draw_hex_grid(space, all_agent_data, agent_portrayal) | ||
case NetworkGrid(): | ||
return _draw_network_grid(space, all_agent_data, agent_portrayal) | ||
case ContinuousSpace() | mesa.experimental.continuous_space.ContinuousSpace(): | ||
return _draw_continuous_space(space, all_agent_data, agent_portrayal) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if the agent_portrayal
was already used to get the agent data, why is it passed into these functions separately?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In mpl_space_drawing we follow similar structure ,that is why passed it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think you have to follow that here, passing parameters that are not being used in the function is more confusing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@nissu99 Can you make the changes here as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Sahil-Chhoker tests/test_solara_viz.py::test_call_space_drawer is failing due to absence of argument.
should i make changes to the tests?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for matplotlib, the reasonagent_portrayal
is passed to the respective _draw functions is because the data from the agents is gathered in the respective draw methods. My suggestion is to do the same on the altair side. So, I suggest moving the gathering of agent data into the respective draw functions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for matplotlib, the reason
agent_portrayal
is passed to the respective _draw functions is because the data from the agents is gathered in the respective draw methods.
I was aware of that, but thought that the new agent data collector function was part of the new API redesign, since it separates the drawing of agents from drawing of grids, is that not the case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fair question. I am not sure yet. I am currently inclined to have some kind of adapter object in between the model and the drawing. However, implementing this will require wider changes throughout the code base. So, the more similar altair and matplotlib drawing are, the easier it will be to make the changes on both sides.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
makes sense!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice idea! this way it will be easier if we add other visualization support.
def _draw_discrete_grid(space, all_agent_data, agent_portrayal): | ||
"""Create Altair visualization for Discrete Grid.""" | ||
invalid_tooltips = ["color", "size", "x", "y"] | ||
x_y_type = "ordinal" | ||
|
||
x_y_type = "ordinal" if not isinstance(space, ContinuousSpace) else "nominal" | ||
encoding_dict = { | ||
"x": alt.X("x", axis=None, type=x_y_type), | ||
"y": alt.Y("y", axis=None, type=x_y_type), | ||
"tooltip": [ | ||
alt.Tooltip(key, type=alt.utils.infer_vegalite_type_for_pandas([value])) | ||
for key, value in all_agent_data[0].items() | ||
if key not in invalid_tooltips | ||
], | ||
} | ||
|
||
has_color = "color" in all_agent_data[0] | ||
if has_color: | ||
encoding_dict["color"] = alt.Color("color", type="nominal") | ||
has_size = "size" in all_agent_data[0] | ||
if has_size: | ||
encoding_dict["size"] = alt.Size("size", type="quantitative") | ||
|
||
chart = ( | ||
alt.Chart( | ||
alt.Data(values=all_agent_data), encoding=alt.Encoding(**encoding_dict) | ||
) | ||
.mark_point(filled=True) | ||
.properties(width=280, height=280) | ||
) | ||
|
||
if not has_size: | ||
length = min(space.width, space.height) | ||
chart = chart.mark_point(size=30000 / length**2, filled=True) | ||
|
||
return chart |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The scale of grid automatically adjusts to the position of agents, is there no way to stop the resizing of the grid? Also no visualization of grid lines.
Some images for better understanding:
Notice the scale and positioning of the larger agent.
Also the coordinates of grids are reversed for matplotlib and Altair, not sure how it can affect the usage.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think resizing is a default behaviour in altair , these things can be handled in the post_process function @quaquel what do you think?
def _draw_hex_grid(space, all_agent_data, agent_portrayal): | ||
"""Create Altair visualization for Hex Grid.""" | ||
invalid_tooltips = ["color", "size", "x", "y", "q", "r"] | ||
x_y_type = "quantitative" | ||
|
||
# Parameters for hexagon grid | ||
size = 1.0 | ||
x_spacing = math.sqrt(3) * size | ||
y_spacing = 1.5 * size | ||
|
||
# Calculate x, y coordinates from axial coordinates | ||
for agent_data in all_agent_data: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same problem as draw_discrete_grid
here. Also no grid lines visualization.
I have not yet implemented the grid visualization |
@quaquel @Sahil-Chhoker I think we should keep the core API very concise, and changes like controlling grid visibility can be handled in the post-processing stage, what is your opinion? |
I also think the the same and I think it is followed everywhere in the visualization API, but the grid resizing every time the agents move may be confusing and not very elegant. |
Grid visibility is not trivial for networks or hexgrids. Hence, it is included on the matplotlib side and controlled via a boolean. As indicated in #2642, I am considering changing the API to give the user more control over this beyond just a boolean. So I am fine with leaving this out of altair for now. I agree with @Sahil-Chhoker, the view limits should be fixed and not change every step. |
@Sahil-Chhoker can you share the example you are using? |
Don't mind the ugly and commented code, I keep changing this example as per need, and I really recommend the Command Center from #2674 for better debugging. import random
from mesa.experimental.cell_space.grid import HexGrid, OrthogonalMooreGrid
from mesa.experimental.continuous_space import ContinuousSpace, ContinuousSpaceAgent
from mesa import Model
import math
from mesa.experimental.cell_space import CellAgent
from mesa.visualization import SolaraViz, make_space_component
from mesa.experimental.cell_space.property_layer import PropertyLayer
import numpy as np
from portrayal_components import PropertyLayerPortrayal, AgentPortrayalStyle, PropertyLayerStyle
import shutil
from agents import myAgent
class myModel(Model):
def __init__(self, seed=None, width=5, height=5 , n_agents=4):
super().__init__(seed=seed)
self.width = width
self.height = height
self.num_agents = n_agents
self.grid = OrthogonalMooreGrid(
(self.width, self.height), capacity=math.inf, torus=False, random=self.random
)
self.x = 1
self.test_layer = PropertyLayer("test", (self.width, self.height), default_value=1, dtype=int)
self.test_layer.data = np.random.randint(2, 6, (self.width, self.height))
for i in range(self.width * self.height):
cell = self.grid.all_cells.cells[i]
if cell.coordinate == (1, 1):
myAgent(self, cell=cell, id=0)
# else:
# myAgent(self, cell=cell, id=self.x)
# self.x += 1
# myAgent(self, cell=self.grid.all_cells.cells[i], id=self.x)
# self.x += 1
# self.grid.add_property_layer(self.test_layer)
# myAgent(self, cell=self.grid.all_cells.cells[17], id=self.x)
# self.x += 1
# myAgent(self, cell=self.grid.all_cells.cells[5], id=self.x)
# self.x += 1
# myAgent(self, cell=self.grid.all_cells.cells[7], id=self.x)
# self.x += 1
# myAgent(self, cell=self.grid.all_cells.cells[1], id=self.x)
# self.x += 1
# myAgent(self, cell=self.grid.all_cells.cells[1], id=self.x)
# self.x += 1
# myAgent(self, cell=self.grid.all_cells.cells[1], id=self.x)
# self.x += 1
# myAgent(self, cell=self.grid.all_cells.cells[1], id=self.x)
self.running = True
def step(self):
self.agents.shuffle_do("step")
class ContModel(Model):
def __init__(self, seed=None):
super().__init__(seed=seed)
self.grid = ContinuousSpace([[0, 1], [0, 1]], torus=True, random=self.random)
self.test_layer = PropertyLayer("test2", [1, 1], default_value=0.0, dtype=float)
# self.grid.add_property_layer(self.test_layer)
self.k_agent = ContinuousSpaceAgent(self.grid, self)
self.k_agent.position = [0.5, 0.5]
self.a_agent = ContinuousSpaceAgent(self.grid, self)
self.a_agent.position = [0.55, 0.55]
def step(self):
self.k_agent.position = [random.random(), random.random()]
def agent_portrayal(agent):
portrayal = AgentPortrayalStyle(
color='tab:orange',
marker='^',
size=150 if agent.id == 0 else 50,
zorder=1
)
return dict(portrayal)
property_layer_portrayal = PropertyLayerPortrayal()
property_layer_portrayal.add_layer(name="test",color="blue", alpha=0.5, colorbar=True, vmin=0, vmax=6)
property_layer_portrayal = dict(property_layer_portrayal)
def post_process(ax):
ax.set_aspect("equal")
ax.get_figure().set_size_inches(8, 8)
model_params = {
"width": {
"type": "SliderInt",
"value": 50,
"label": "Width",
"min": 5,
"max": 60,
"step": 1,
},
"height": {
"type": "SliderInt",
"value": 50,
"label": "Height",
"min": 5,
"max": 60,
"step": 1,
},
}
model = myModel()
space_component = make_space_component(
agent_portrayal=agent_portrayal, backend="altair", draw_grid=True
)
# space_component = make_space_component(
# agent_portrayal=agent_portrayal, draw_grid=True, post_process=post_process, propertylayer_portrayal=property_layer_portrayal
# )
page = SolaraViz(
model,
components=[space_component],
model_params=model_params,
name="Test Model",
additional_imports={
"myAgent": myAgent,
}
)
page |
you are using some custom imports here ,nevermind Thank You! |
for more information, see https://pre-commit.ci
@Sahil-Chhoker can you check the changes? |
@nissu99 Before I do an through review, can you please tell me the scope of this PR and what it aims to add into the project in detail except the ones listed in the description. And if all the functionalities are there or not. |
This PR added Altair Plot support ,worked on a generic data extracting function , created different functions for drawing different types of grids similar to that in matplot _space and tried to make overall api as similar as that of mpl_space api. |
@nissu99 very sorry for misleading you but the generic agent data collection needs to be changed to be like the one used in matplotlib. |
No worries ,this is how we learn. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have done the review of basic rectangular and hex grids, first I would love to get them finished up. Then I also noticed that the implementation of the new Network is also missing. I will do a review again once you make the changes, please take a look at the review and don't hesitate to ask for implementations as well if unable to get them.
def _draw_discrete_grid(space, all_agent_data, agent_portrayal): | ||
"""Create Altair visualization for Discrete Grid.""" | ||
invalid_tooltips = ["color", "size", "x", "y"] | ||
x_y_type = "ordinal" | ||
|
||
x_y_type = "ordinal" if not isinstance(space, ContinuousSpace) else "nominal" | ||
encoding_dict = { | ||
"x": alt.X("x", axis=None, type=x_y_type), | ||
"y": alt.Y("y", axis=None, type=x_y_type), | ||
"tooltip": [ | ||
alt.Tooltip(key, type=alt.utils.infer_vegalite_type_for_pandas([value])) | ||
for key, value in all_agent_data[0].items() | ||
if key not in invalid_tooltips | ||
], | ||
} | ||
|
||
has_color = "color" in all_agent_data[0] | ||
if has_color: | ||
encoding_dict["color"] = alt.Color("color", type="nominal") | ||
has_size = "size" in all_agent_data[0] | ||
if has_size: | ||
encoding_dict["size"] = alt.Size("size", type="quantitative") | ||
|
||
chart = ( | ||
alt.Chart( | ||
alt.Data(values=all_agent_data), encoding=alt.Encoding(**encoding_dict) | ||
) | ||
.mark_point(filled=True) | ||
.properties(width=280, height=280) | ||
) | ||
|
||
if not has_size: | ||
length = min(space.width, space.height) | ||
chart = chart.mark_point(size=30000 / length**2, filled=True) | ||
|
||
chart = chart.encode( | ||
x=alt.X( | ||
"x", axis=None, type=x_y_type, scale=alt.Scale(domain=(0, space.width - 1)) | ||
), | ||
y=alt.Y( | ||
"y", axis=None, type=x_y_type, scale=alt.Scale(domain=(0, space.height - 1)) | ||
), | ||
) | ||
|
||
return chart |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
case Grid(): | ||
all_agent_data = _get_agent_data_new_discrete_space(space, agent_portrayal) | ||
return _draw_discrete_grid(space, all_agent_data) | ||
case _Grid(): | ||
all_agent_data = _get_agent_data_old__discrete_space(space, agent_portrayal) | ||
case ContinuousSpace(): | ||
all_agent_data = _get_agent_data_continuous_space(space, agent_portrayal) | ||
return _draw_legacy_grid(space, all_agent_data) | ||
case HexGrid(): | ||
return _draw_hex_grid(space, all_agent_data) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This order will never draw the HexGrid
, because the hex grid inherits form the Grid. I think reversing their order will fix it.
case Grid(): | ||
# New DiscreteSpace or experimental cell space | ||
for cell in space.all_cells: | ||
for agent in cell.agents: | ||
data = agent_portrayal(agent) | ||
data.update({"x": cell.coordinate[0], "y": cell.coordinate[1]}) | ||
all_agent_data.append(data) | ||
|
||
Returns: | ||
list of dicts | ||
case _Grid(): | ||
# Legacy Grid | ||
for content, (x, y) in space.coord_iter(): | ||
if not content: | ||
continue | ||
agents = [content] if not hasattr(content, "__iter__") else content | ||
for agent in agents: | ||
data = agent_portrayal(agent) | ||
data.update({"x": x, "y": y}) | ||
all_agent_data.append(data) | ||
|
||
case HexGrid(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Again this order won't work, because the HexGrid
inherits from Grid.
|
||
case HexGrid(): | ||
# Hex-based grid | ||
for content, (q, r) in space.coord_iter(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
coord_iter
doesn't exist for HexGrid
def _draw_hex_grid(space, all_agent_data, agent_portrayal): | ||
"""Create Altair visualization for Hex Grid.""" | ||
invalid_tooltips = ["color", "size", "x", "y", "q", "r"] | ||
x_y_type = "quantitative" | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Sahil-Chhoker Sorry for the inconvenience. I will test more carefully this time. |
#2435
@quaquel @EwoutH I’ve added a new component, make_altair_plot_component. Could you kindly check if it aligns well with the project?
P.S working on space support.