Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add dependencies to allow for HandBrake QSV support #356

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

add dependencies to allow for HandBrake QSV support #356

wants to merge 2 commits into from

Conversation

DZaremba14600
Copy link

This is my first git/github submission ever.

I apologize if I miss any protocols, my intent is not to irritate.

I tried to comment appropriately, if its too much/not enough just let me know.

If the project maintainers would prefer other protocols or policies be followed, just let me know.

An earlier version of these changes was used by me to process close to 60 BluRay movies and they processed very quickly (less than 2 hours per disk average over parts of 4 days (on 2 machines) and without error. I tweaked the code, cleaned up the comments, and ran several Docker Image builds to make sure there weren't any errors in the builds. I had to get up on how git worked. So here's my submission.

Added support for Handbrake to support use of QSV.

The only file changed is scripts/install_mkv_hb_deps.sh

Added code to add repository for Intel graphics GPU support. This will allow latest packages to be pulled.
Added an update/upgrade step to get latest packages index list and upgrade all existing packages.
Added step to install packages for OneVPL Compute Runtime, Media, and Mesa packages, misc utils, for QSV use in HandBrake.
Added step to install specific QSV packages as called for by HandBrake's website.
Added step to install new HandBrake version 1.9 dependencies. These seem identical to the previously included HandBrake dependencies installed at line 21. Identical packages will be upgrade/skipped; any different packages will be installed (this being the case, line 21 could actually be removed/merged with line 78. I decided to code this way to ensure everything is shown to be working through widespread use, and in the next update these redundant lines could be merged or line 21 removed/replaced.
Added step to include GTK library packages (used in HandBrake GUI). Even though this is CLI only, there is functionality in place to allow easy use of custom PRESETS on the handbrake args line using .json files exported from the GUI version. During testing there seemed to be issues unless these were included. They can be removed if further use show them to be unnecessary.

Testing shows this builds and the Docker Container comes up and runs. Obviously I didn't test every function of ARM, but I did run ARM and processed the BluRay of 'Unforgiven', using latest Handbrake 1.9, a custom preset that utilizes QSV, and the job completed without errors.

Thank you.

@microtechno9000
Copy link
Collaborator

Thanks for the first commit

There are no specific rules setup for the ARM dependencies

I will check this on my machine to check it doesn't break containers with no QSV cards.

Wont get to this till after Christmas

Thanks for helping the community

@microtechno9000 microtechno9000 self-assigned this Dec 20, 2024
@DZaremba14600
Copy link
Author

DZaremba14600 commented Dec 21, 2024 via email

@microtechno9000 microtechno9000 added enhancement New feature or request dependencies Pull requests that update a dependency file labels Dec 26, 2024
@microtechno9000
Copy link
Collaborator

microtechno9000 commented Dec 28, 2024 •
edited
Loading

Pulled this PR and ran a quick test, the new image takes a little longer to build and is ~700MB larger, which is ok, noting users are ripping DVD and Bluray images 10x as large.

Initial test on my non QSV machine, the rip appeared to take over a day for a DVD, which would typically take 4-5 hours (not the fastest machine). Need to re-run the test again because they were different discs.
However, if enabling QSV on a non-supported system increases the ripping time 2-3 times that is definitely not ideal.

$ docker image ls
REPOSITORY                                 TAG           IMAGE ID       CREATED          SIZE
automaticrippingmachine/arm-dependencies   1.2.2         2ec395155423   2 days ago       3.3GB
automaticrippingmachine/arm-dependencies   1.2.1         5172bfb8ec35   2 days ago       2.63GB

@DZaremba14600
Copy link
Author

DZaremba14600 commented Dec 30, 2024 via email

@ReubenBTalbott
Copy link

Any update on this? I recently got an Arc GPU and would love to be able to use it to encode everything in AV1.

@DZaremba14600
Copy link
Author

Sorry for the delay on this. Holidays and then work.

I'll be responding to this in the next day or so.

I'm going to pull the lastest ARM, build the Docker image. Add in my updates, build a 2nd Docker image, and then run a test run between the two - I'll do a DVD, a Bluray, and a UHD disk both with and without QSV. I'll log the results and submit that.

I realize the Docker image will be larger, it was to be, its installing all the packages to support Handbrake using QSV. The time to rip the disk should not change, that's MakeMKV and I'm not touching that, and I'm not changing anything in the actual Handbrake build either. I'm installing a set of packages to allow Handbrake to use QSV, that's it..

I see that processing a DVD took longer. I actually have a DVD I need to rip. SO I'll be doing a run as described. All of this will be done on the same system, no hardware will change.

All I can say is that I can rip and process a Bluray with a with Handbrake setting of Ultra High Quality, CRF set to 18, and I can do the job, from inserting the disk to completion in about an hour. Its using a Sparkle Intel ARC A310 to transcode and it flies. I've done a few AV1 and they fly too.

Let me run this and I'll report back hopefully tomorrow or Wednesday.

Again, sorry for my recent absence.

@microtechno9000
Copy link
Collaborator

No issues, we all have lives and this is open source so people are contributing for free.

Thanks for understanding and running the tests

@DZaremba14600
Copy link
Author

Well, an update and a question.

I wanted to run a test suite for this QSV enhancement. I was particularly curious as to what may have been causing an issue with processing DVD's. It was reported that the proposed enhancements actually made DVD processing slower. Very odd. The test suite I set up was this:

One container using the latest ARM Docker code. I would clone the repository and build the Docker image unchanged.
A second container using the latest Docker code including the QSV enhancements. This is the container my pull request was for.

Using the stock Docker container, I would process a Bluray disk, an UHD4K disk, and a DVD movie disk using a non-QSV preset. Use the logs for timing baseline and other info.

Using the QSV-enabled Docker container, I would process a Bluray disk, a UHD4K disk, and a standard DVD movie disk in two sets, one using the same non-QSV preset as the stock container, and then the same preset, but using a corresponding QSV encoder, with all else being identical. Basically stock non-QSV enabled container with non-QSV preset, QSV-enabled container with non-QSV preset, and QSV-enable container with QSV-enabled preset.

I figure that should give a good comparison between a stock baseline and a QSV-enabled baseline.

I was going to present a matrix of the relative information for these 6 combinations.

Everything seems to work fine EXCEPT when I try to process a DVD using QSV enabled h.265 HEVC encoder. The job rips the DVD but bombs almost instantly when the Handbrake processing begins. It even bombs Handbrake when I try to process the DVD MKV manually using standalone Handbrake installed on the host. It seems to process fine except when I select QSV h.265 HEVC.

Any thoughts? There is an error thrown in the logs, I'll try to post it tonight.

I've read that h.265 HEVC is not really good with DVD material. I've read its better suited for HD material. I tried QSV h.264 codec and that seemed to start running ok. I did not let it complete.

When I say bombs, its almost instantly terminates.

Any DVD experiences that might be helpful? With HD Blurays, HD and 4K TV's I wasn't personally planning on processing a lot of DVD material, but I know others will.

The Bluray and HDR4K disks process fine in all instances and the QSV processing is fast.

Any thoughts or suggestions?

I'll try to post the errors in the logs for the DVD processing that failed. I'd like to give this a final push Sunday and Monday if I hear back,

Any input would be appreciated. Just let me know.

Thanks in advance and thanks for your patience.

@microtechno9000
Copy link
Collaborator

microtechno9000 commented Feb 25, 2025 •
edited
Loading

Thanks for running those tests.

I am no Handbrake expert, maybe @1337-server or @shitwolfymakes could help?

@DZaremba14600
Copy link
Author

Update -

The pull request I submitted originally for this QSV support utilized the latest Handbrake at that time, which was 1.9.0. That's when I began testing DVD's, up to that point I was only testing with Bluray and 4K disks, and was seeing no issues. Testing against DVD's was where I was seeing issues. That didn't make a lot of sense. Even manually running Handbrake using a QSV-enabled preset against an MKV ripped straight from a DVD was terminating almost instantly. The logs indicated it was when an audio stream was being processed. I suspected it might be something with HandBrake or even with ffmpeg.

Low and behold I was the other day a newer Handbrake was released, version 1.9.2 and it specifically mentions improvements with ffmpeg processing (although only E-AC3 streams were specifically mentioned in the info I saw.

Anyway, last night I rebuilt the newer version HandBrake from source on my host (as a baseline), I sync'd my fork of ARM and the ARM-dependencies, I sync's my local copy of the forks on my system, I deleted the ARM and ARM-dependencies images in Docker and rebuilt my docker images so I'd have the latest HandBrake used as part of the build.

I currently have an ARM Docker image that is 'stock' and using the lastest HandBrake. This will act as a control and a baseline.
I also have an ARM Docker image that includes the QSV-enabled enhancements I've added that uses the latest HandBrake. This image will be used to test non-qsv preset jobs against the stock image using the same non-qsv preset (they should run identically). I also will then run qsv preset jobs against the stock image using non-qsv presets.

This should give me a good matrix of results as for performance and reliablity. I'll be running DVD, Bluray, and 4K disk jobs.

Tonight I'll start by running Handbrake on an MKV ripping from a DVD on the host to make sure that generic presets don't terminate like they were. If those succeed, I'll assume there was some issue with the upgrade from 1.8.x to 1.9.0 and then proceed to run tests using ARM in these containers.

I'll report the results and the test bed.

I wanted to update this thread since I've been tinkering with this without report on the thread.

I'll let everyone know how it went.

Thanks.

@DZaremba14600
Copy link
Author

Oh boy, spoke too soon -

I just noticed Intel updated their website with regards to packages to install for Ubuntu 22.04 for GPU support and corresponding compute-related and media-related packages. Of note is that the number of packages has been significantly decreased. Perhaps there has been some streamlining and consolidation. Hopefully that also means smaller images.

I'll make appropriate changes, rebuild the images and verify that the encoder support is there. If that's the case, I'll go ahead with the testing.

Fingers crossed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@microtechno9000 microtechno9000

Labels
dependencies Pull requests that update a dependency file enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@DZaremba14600 @microtechno9000 @ReubenBTalbott