Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sort metrics table by combined performance score (CPS) with radar chart to dynamically adjust metric weights #223

Merged
merged 5 commits into from
Mar 16, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@

Please check the following items before submitting your PR:

- [ ] I have created a new folder and YAML metadata file `models/<arch_name>/<model_variant>.yml` for my submission. `arch_name` is the name of the architecture and `model_variant.yml` includes things like author details, training set names and/or important hyperparameters.
- [ ] I have created a new folder and YAML metadata file `models/<arch_name>/<model_variant>.yml` for my submission. `arch_name` is the name of the architecture and `model_variant.yml` includes things like author details, training set names and important hyperparameters.
- [ ] I have added the my new model as a new attribute on the [`Model.<arch_name>` enum](https://github.com/janosh/matbench-discovery/blob/57d0d0c8a14cd317/matbench_discovery/enums.py#L274) in `enums.py`.
- [ ] I have uploaded the energy/force/stress model prediction file for the WBM test set to Figshare or another cloud storage service (`<yyyy-mm-dd>-<arch_name>-preds.csv.gz`).
- [ ] I have uploaded the energy/force/stress model prediction file for the WBM test set to Figshare or another cloud storage service (`<yyyy-mm-dd>-<model_variant>-preds.csv.gz`).
- [ ] I have uploaded the model-relaxed structures file to Figshare or another cloud storage service (`<yyyy-mm-dd>-wbm-IS2RE-FIRE.json.gz`).
- [ ] I have uploaded the phonon predictions to Figshare or another cloud storage service (`<yyyy-mm-dd>-kappa-103-FIRE-<values-of-dist|fmax|symprec>.gz`).
- [ ] I have included the urls to the Figshare files in the YAML metadata file (`models/<arch_name>/<model_variant>.yml`). If not using Figshare I have included the urls to the cloud storage service in the description of the PR.
Expand Down
18 changes: 18 additions & 0 deletions models/sevennet/sevennet-mf-ompa.yml
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,24 @@ metrics:
pred_file: models/sevennet/sevennet-mf-ompa/2025-03-11-wbm-geo-opt-FIRE.json.gz
pred_file_url: https://figshare.com/files/52983491
struct_col: sevennet_structure
symprec=1e-2:
rmsd: 0.0115 # Å
n_sym_ops_mae: 1.7053 # unitless
symmetry_decrease: 0.0467 # fraction
symmetry_match: 0.8181 # fraction
symmetry_increase: 0.128 # fraction
n_structures: 256963 # count
analysis_file: models/sevennet/sevennet-mf-ompa/2025-03-11-wbm-geo-opt-FIRE-symprec=1e-2-moyo=0.4.2.csv.gz
analysis_file_url: https://figshare.com/files/53029115
symprec=1e-5:
rmsd: 0.0115 # Å
n_sym_ops_mae: 2.0326 # unitless
symmetry_decrease: 0.0439 # fraction
symmetry_match: 0.7057 # fraction
symmetry_increase: 0.2453 # fraction
n_structures: 256963 # count
analysis_file: models/sevennet/sevennet-mf-ompa/2025-03-11-wbm-geo-opt-FIRE-symprec=1e-5-moyo=0.4.2.csv.gz
analysis_file_url: https://figshare.com/files/53029142
discovery:
pred_file: models/sevennet/sevennet-mf-ompa/2025-03-11-wbm-IS2RE.csv.gz
pred_file_url: https://figshare.com/files/52983488
Expand Down
2 changes: 1 addition & 1 deletion site/src/lib/GeoOptMetricsTable.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@
{#snippet cell({ col, val })}
{#if col.label === `Links` && val}
{@const links = val}
{#each links.files as { url: href, title, icon } (href)}
{#each links.files as { url: href, title, icon } (title + href)}
{#if href}
<a {href} {title} target="_blank" rel="noopener noreferrer">
{@html icon}
Expand Down
100 changes: 79 additions & 21 deletions site/src/lib/HeatmapTable.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,18 @@
import { titles_as_tooltips } from 'svelte-zoo/actions'
import { flip } from 'svelte/animate'
import { writable } from 'svelte/store'
import type { HeatmapColumn } from './types'

type CellVal = string | number | undefined | null
type RowData = Record<string, CellVal>
type TableData = RowData[]
import type { CellVal, HeatmapColumn, RowData, TableData } from './types'

interface Props {
data: TableData
columns?: HeatmapColumn[]
sort_hint?: string
style?: string | null
cell?: Snippet<[{ row: RowData; col: HeatmapColumn; val: CellVal }]>
controls?: Snippet
initial_sort_column?: string
initial_sort_direction?: `asc` | `desc`
fixed_header?: boolean
}

let {
Expand All @@ -28,9 +28,19 @@
sort_hint = `Click on column headers to sort table rows`,
style = null,
cell,
controls,
initial_sort_column,
initial_sort_direction,
fixed_header = false,
}: Props = $props()

const sort_state = writable({ column: ``, ascending: true })
// Add container reference for binding
let container: HTMLDivElement

const sort_state = writable({
column: initial_sort_column || ``,
ascending: initial_sort_direction !== `desc`,
})

let clean_data = $state(data)
$effect(() => {
Expand Down Expand Up @@ -70,13 +80,27 @@
})
}

function calc_color(value: number | string | undefined, col: HeatmapColumn) {
if (col.color_scale === null || typeof value !== `number`)
function calc_color(value: number | string | undefined | null, col: HeatmapColumn) {
// Skip color calculation for null values or if color_scale is null
if (
value === null ||
value === undefined ||
col.color_scale === null ||
typeof value !== `number`
) {
return { bg: null, text: null }
}

const col_id = get_col_id(col)
const values = clean_data.map((row) => row[col_id])
const range = [min(values) ?? 0, max(values) ?? 1]
const numericValues = clean_data
.map((row) => row[col_id])
.filter((val): val is number => typeof val === `number`) // Type guard to ensure we only get numbers

if (numericValues.length === 0) {
return { bg: null, text: null }
}

const range = [min(numericValues) ?? 0, max(numericValues) ?? 1]
if (col.better === `lower`) {
range.reverse()
}
Expand Down Expand Up @@ -116,14 +140,28 @@
}
</script>

<div class="table-container" {style}>
<table use:titles_as_tooltips>
<!-- Table header with sort hint and controls side by side -->
<div class="table-header">
{#if Object.keys($sort_state).length && sort_hint}
<div class="sort-hint">{sort_hint}</div>
{/if}

<!-- Add controls rendering here -->
{#if controls}
<div class="controls-container">
{@render controls()}
</div>
{/if}
</div>

<div bind:this={container} class="table-container" {style}>
<table use:titles_as_tooltips class:fixed-header={fixed_header} class="heatmap">
<thead>
<!-- Don't add a table row for group headers if there are none -->
{#if visible_columns.some((col) => col.group)}
<!-- First level headers -->
<tr class="group-header">
{#each visible_columns as {label, group, tooltip} (label + group)}
{#each visible_columns as { label, group, tooltip } (label + group)}
{#if !group}
<th></th>
{:else}
Expand Down Expand Up @@ -169,13 +207,13 @@
style:background-color={color.bg}
style:color={color.text}
style={col.style}
title={[undefined, null].includes(val) ? `not available` : null}
title={typeof val === `undefined` || val === null ? `not available` : null}
>
{#if cell}
{@render cell({ row, col, val })}
{:else if typeof val === `number` && col.format}
{pretty_num(val, col.format)}
{:else if [undefined, null].includes(val)}
{:else if val === undefined || val === null}
n/a
{:else}
{@html val}
Expand All @@ -191,15 +229,11 @@
<style>
.table-container {
overflow-x: auto;
max-width: 100%;
scrollbar-width: none;
margin: auto;
font-size: var(--heatmap-font-size, 0.9em);
}

/* https://stackoverflow.com/a/38994837 */
.table-container {
/* https://stackoverflow.com/a/38994837 */
scrollbar-width: none; /* Firefox */
max-width: 90vw;
}
.table-container::-webkit-scrollbar {
display: none; /* Safari and Chrome */
Expand Down Expand Up @@ -247,4 +281,28 @@
border-bottom: 1px solid black;
text-align: center;
}

/* Styles for the table header with sort hint and controls */
.table-header {
display: flex;
align-items: center;
justify-content: space-between;
margin-bottom: 0.5rem;
flex-wrap: wrap;
gap: 0.5rem;
padding: 0.25rem 0;
border-bottom: 1px solid rgba(255, 255, 255, 0.1);
}

.sort-hint {
font-size: 0.85em;
color: var(--text-muted, #aaa);
margin: 0;
}

.controls-container {
display: inline-flex;
align-items: center;
margin-left: auto;
}
</style>
Loading
Loading