split integration test execution #632
Conversation
There was a problem hiding this comment.
Left a few comments, but overall I really like this, especially the virtiofsd part! Thanks for working on this.
I'm not entirely familiar with all the specifics of the matrix strategy on GitHub, but it seems really cool. I think we should go ahead and merge this to give it a try.
| /usr/lib/virtiofsd | ||
| key: virtiofsd-binary | ||
| - if: ${{ steps.cache-virtiofsd.outputs.cache-hit != 'true' && steps.cache-kernel.outputs.cache-hit != 'true' }} | ||
| run: cargo install virtiofsd && sudo cp -a ~/.cargo/bin/virtiofsd /usr/lib/ |
There was a problem hiding this comment.
Ahh, really nice! I was trying to figure out a way to add virtiofsd, it was easier than expected! :)
.github/workflows/caching-build.yml
Outdated
| prefix-key: "1" | ||
| - run: cargo build --manifest-path rust/${{ matrix.component }}/Cargo.toml | ||
| - run: cargo test --manifest-path rust/${{ matrix.component }}/Cargo.toml --no-run | ||
| - run: vng -v --memory 10G --cpu 8 --force-9p -r linux/arch/x86/boot/bzImage --net user -- cargo test --manifest-path rust/${{ matrix.component }}/Cargo.toml |
There was a problem hiding this comment.
I think we should drop --force-9p also from here.
There was a problem hiding this comment.
lol ofc I missed the instance of that in the file I was editing the most, thx
There was a problem hiding this comment.
ahh forgot rg skips hidden directories and .github is one, thx.
.github/workflows/caching-build.yml
Outdated
| - run: vng -v --memory 10G --cpu 8 --force-9p -r linux/arch/x86/boot/bzImage --net user -- cargo test --manifest-path rust/Cargo.toml | ||
| - run: cargo build --manifest-path scheds/rust/${{ matrix.scheduler }}/Cargo.toml | ||
| - run: cargo test --manifest-path scheds/rust/${{ matrix.scheduler }}/Cargo.toml --no-run | ||
| - run: vng -v --memory 10G --cpu 8 --force-9p -r linux/arch/x86/boot/bzImage --net user -- cargo test --manifest-path scheds/rust/${{ matrix.scheduler }}/Cargo.toml |
There was a problem hiding this comment.
Ditto, now that we have virtiofsd we don't need --force-9p.
likewhatevs
force-pushed
the
split-integration-test
branch
from
9fa8feb
to
d14e96e
Compare
split build and test jobs to reduce ci turnaround time and make it clear what is failing when something fails. also add virtiofsd to deps to make test compilation faster (most test time is compliation) and remove all force 9ps.
likewhatevs
force-pushed
the
split-integration-test
branch
from
d14e96e
to
120211d
Compare
split build and test jobs to reduce ci turnaround time
and make it clear what is failing when something fails.
also add virtiofsd to deps to make test compilation faster
(most test time is compliation) and remove all force 9ps.
also have rust tests on schedulers be ran.
time w/ no caches is 23 minutes, time with caches is 7 minutes:
https://github.com/likewhatevs/scx/actions/runs/10756353007
i think this should address GH-462, it goes as far as the trade off's involved make sense imo