[Proposal] Add MediaElement as TextureView in Android

[jonmdev]

Feature name

MediaElement as TextureView in Android

Link to discussion

#1891

Progress tracker

• Android Implementation
• iOS Implementation
• MacCatalyst Implementation
• Windows Implementation
• Tizen Implementation
• Unit Tests
• Samples
• Documentation

Summary

It is now 11 months since I raised the issue of MediaElement not supporting TextureView here for Android:

#1773

And 9 months since everyone seemed to agree this should be added:

#1891

The implementation of adding this is very simple (or at least it was back when I last looked at it).

My workaround 9 months ago was to add an xml to Platforms/Android/Resources/layout.textureview.xml:

<?xml version="1.0" encoding="utf-8" ?>
<com.google.android.exoplayer2.ui.StyledPlayerView
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    android:id="@+id/test"
    app:surface_type="texture_view"
	app:shutter_background_color="#FFFFFF"
    android:layout_width= "match_parent"
    android:layout_height= "match_parent" 
	/>

Then in MediaManager.Android.cs we could do:

	public (PlatformMediaElement platformView, StyledPlayerView PlayerView) CreatePlatformView()
	{
		ArgumentNullException.ThrowIfNull(MauiContext.Context);
		Player = new IExoPlayer.Builder(MauiContext.Context).Build() ?? throw new NullReferenceException();
		Player.AddListener(this);
		InitializeMediaSession();

        //CUSTOM CODE HERE: 
        XmlReader xmlResource = Microsoft.Maui.ApplicationModel.Platform.CurrentActivity.Application.Resources.GetXml(Microsoft.Maui.Resource.Layout.textureview);
        xmlResource.Read();
        Android.Util.IAttributeSet attributes = Android.Util.Xml.AsAttributeSet(xmlResource);

		PlayerView = new StyledPlayerView(MauiContext.Context, attributes)
		{
			
			Player = Player,
			UseController = false,
			ControllerAutoShow = false,
			LayoutParameters = new RelativeLayout.LayoutParams(ViewGroup.LayoutParams.MatchParent, ViewGroup.LayoutParams.MatchParent)
		};

        System.Diagnostics.Debug.WriteLine("SURFACE TYPE " + PlayerView.VideoSurfaceView.GetType());
        xmlResource.Dispose();

In other words we just need to be able to manually pass in some attribute xml into the StyledPlayerView constructor.

I worked around this issue by copying and pasting the entire MediaElement out of Community Toolkit. The code above is all I had to change to make it a TextureView, as again this is intended by Google to be a very simple thing. This has served fine as a temporary solution. But it is a timebomb for me as now everything from that old version is becoming obsoleted.

The addition of TextureView support is necessary for virtually any normal layout functions in Android as noted in the prior two discussions.

Is there any way to add this to the new version of Community Toolkit?

@ne0rrmatrix I believe claimed he knew a way to make the TextureView without needing any xml, but I could not figure out any other way to do it. I am not sure if these constructors have changed also since then. I figure it is worth reassessing and seeing what the solution is if we can still fix this somehow.

Thanks for any support on this.

Motivation

Android Exoplayer can be rendered as a SurfaceView (which does not respect layout like overlapping objects or masking) or a TextureView (which does respect layout like overlapping objects and masking).

Normal modern layouts (eg. with rounded corners or overlapping elements) will need TextureView to integrate video into those views.

Detailed Design

As above, Exoplayer typically just needs the TextureView passed as an argument on construction. So we need a way to construct MediaElement with an argument that accounts for this.

Usage Syntax

My belief as previously expressed is we should have something like:

class MediaElementConstructionArgs{
    public AndroidViewType androidViewType = AndroidViewType.SurfaceView; //(or AndroidViewType.TextureView)
}
enum AndroidViewType{
    SurfaceView, TextureView
}

As this can be used for other construction parameters later if ever needed and it is platform agnostic (could hold various platform specific configuration elements/enums.

Then in usage we can do:

MediaElementConstructionArgs constructionArgs = new() {androidViewType = AndroidViewType.TextureView };
MediaElement mediaElement = new(constructionArgs);

Pretty simple, logical, and straightforward I think.

Drawbacks

There is no downside. The lack of TextureView is a glaring hole in this project which is demonstrated in the examples I have given. Lacking TextureView prevents the MediaElement from being used in any modern looking interface. It forces us to do crazy workarounds like I described of copying and pasting the entire MediaElement into our code just to change 3-5 lines or we can't make our modern looking layouts work in Android.

Unresolved Questions

I am not knowledgeable or capable of how to modify these open source types of projects on GitHub myself and pushing updates or passing tests, etc. I have never done this before and have no training in doing so. So while I very much hope this will be fixed as I expect my temporary solution to self destruct imminently (forcing me to redo all that manual importing of code again which is a messy and seemingly pointless temporary job), I cannot do much more to fix it myself. Thus I really hope someone who does know how to fix these types of projects could add a simple method like I described or something like it so we can all benefit from a stable and long term fix.

Thanks for any help and all the hard work on the project.

[bijington]

@jonmdev It sounds like you already have the skills to modify the code so how about I help you with the bits that you aren't sure on - how to contribute to open source?

[jonmdev]

@jonmdev It sounds like you already have the skills to modify the code so how about I help you with the bits that you aren't sure on - how to contribute to open source?

Okay, I'm certainly game to give it a shot. I cloned the repository and I am looking into it.

The constructor is still roughly the same in MediaManager.android.cs here:

public (PlatformMediaElement platformView, PlayerView PlayerView) CreatePlatformView()
{
	Player = new ExoPlayerBuilder(MauiContext.Context).Build() ?? throw new InvalidOperationException("Player cannot be null");
	Player.AddListener(this);
	PlayerView = new PlayerView(MauiContext.Context)
	{
		Player = Player,
		UseController = false,
		ControllerAutoShow = false,
		LayoutParameters = new RelativeLayout.LayoutParams(ViewGroup.LayoutParams.MatchParent, ViewGroup.LayoutParams.MatchParent)
	};

The constructor PlayerView has another overload public unsafe PlayerView (global::Android.Content.Context? context, global::Android.Util.IAttributeSet? attrs) : base (IntPtr.Zero, JniHandleOwnership.DoNotTransfer)

This is the one we need (same as in my workaround above). We must pass in the IAttributeSet that will make it a TextureView here.

I researched this extensively when I first studied this 9-11 months ago and I could find no way to dynamically generate the IAttrituteeSet except from an XML file. Every other method I tried failed with errors.

So as noted above, I had to add a textureview.xml to Platforms/Android/Resources/layout.textureview.xml:

<?xml version="1.0" encoding="utf-8" ?>
<com.google.android.exoplayer2.ui.StyledPlayerView
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    android:id="@+id/test"
    app:surface_type="texture_view"
	app:shutter_background_color="#FFFFFF"
    android:layout_width= "match_parent"
    android:layout_height= "match_parent" 
	/>

Then I could pass this in by:

        XmlReader xmlResource = Microsoft.Maui.ApplicationModel.Platform.CurrentActivity.Application.Resources.GetXml(Microsoft.Maui.Resource.Layout.textureview);
        xmlResource.Read();
        Android.Util.IAttributeSet attributes = Android.Util.Xml.AsAttributeSet(xmlResource);

		PlayerView = new StyledPlayerView(MauiContext.Context, attributes)

The world has changed in even 9-11 months and now asking ChatGPT, it says things like:

The simplest and most robust solution is to define your attributes in an XML layout file (even if it’s a minimal one) and inflate it to get the IAttributeSet. This is the intended use case.
Direct Construction of IAttributeSet: Not supported. The interface is meant to be provided by the Android XML inflation mechanism.
Recommended Approach: Use an XML layout file with the appropriate attribute (app:surface_type="texture_view") or, if available, a public property on the view to set the desired behavior.
Custom Implementation: Creating your own IAttributeSet implementation is technically possible but complex and error-prone.

So according to it, this is still the only correct way to do it. Unless @ne0rrmatrix you do know a different or better way?

ChatGPT says I can add the xml anywhere arbitrarily in the MediaElement folder path and then add to the MediaElement csproj like:

<ItemGroup>
  <AndroidResource Include="Platforms\Android\Resources\layout\textureview.xml" />
</ItemGroup>

So I am trying that unless advised otherwise. Seems to be working so far.

[bijington]

@jonmdev thank you! I'll defer to James for the assistance with the media specific bits surrounding xml files, etc. for now I'd like to focus on helping with the general approach to contributing.

Sadly you can't just clone this repository because you don't have permission to push branches. Therefore to save pain later I would like to guide you through the steps you need to follow. They are:

1. Create a fork
2. Clone your fork
3. Create a branch on your fork (reduces pain when merging back in)
4. Make your changes
5. Push your changes
6. Navigate to this repository and it will provide a handy link to create the repository

This also looks pretty helpful https://www.freecodecamp.org/news/how-to-fork-a-github-repository/

[jonmdev]

Oh geez. Okay I'll do all that. I'm already figuring out the xml stuff. I think it's working so far. This shouldn't take more than a day I don't think. I'll copy over to a fork and post back if any issues or when done.

[bijington]

Oh geez. Okay I'll do all that. I'm already figuring out the xml stuff. I think it's working so far. This shouldn't take more than a day I don't think. I'll copy over to a fork and post back if any issues or when done.

Great work! Feel free to reach out if you need anything! Are you on the discord server for the toolkit?

[ne0rrmatrix]

Sorry about the late reply. Yes you correct. Your method is the only way this will work atm. I had a few ideas but they did not work out. So yes you are 100 correct about this.

[jonmdev]

Well I'm dead ended.

1. I forked it.
2. I cloned it.
3. I made a branch.
4. I opened samples/CommunityToolkit.Maui.Sample.sln and added a new sample project from the default Maui project.
5. I right clicked on Dependencies for this new project and added references to some of the CommunityToolkit pieces.
6. I had to add to mauiprogram.cs:

.UseMauiCommunityToolkit()
.ConfigureMauiHandlers(handlers =>
{
	// Explicitly register the MediaElement handler if it's not being auto-registered
	handlers.AddHandler(typeof(MediaElement), typeof(MediaElementHandler));
})

(For some reason I had to manually add the handler or it said it didn't exist.)

I can now build this sample project with a MediaElement mediaElement = new() in App.xaml.cs in Windows but in Android, I get an exception on this existing code:

PlayerView = new PlayerView(MauiContext.Context)
{
	Player = Player,
	UseController = false,
	ControllerAutoShow = false,
	LayoutParameters = new RelativeLayout.LayoutParams(ViewGroup.LayoutParams.MatchParent, ViewGroup.LayoutParams.MatchParent)
};

"Java.Lang.AbstractMethodError: 'abstract method "void androidx.media3.common.Player$Listener.onSurfaceSizeChanged(int, int)"'"

And it breaks.

This is without me even implementing my new code. So already I have two weird behaviors (1) Had to manually add the handlers and (2) Getting exception on existing method.

What did I do wrong?

This is the sort of thing I have no idea about and I am just wasting time with. I can write the TextureView code. I have done it before. Would likely take me a few hours to do all of it. But I have no idea about any of this. Can't add the new method if the existing won't work.

Any help or solution?

https://github.com/jonmdev/CommunityToolkit-Maui/tree/AddAndroidTextureView

[ne0rrmatrix]

Your texture view is referencing media 2 and needs to use AndroidX.Media3.UI.PlayerView in textureview.xml

[ne0rrmatrix]

I put together a quick and dirty sample for you to look at and maybe use if it does what you need? main...ne0rrmatrix:MauiOld:TextureViewIdea

[jonmdev]

I put together a quick and dirty sample for you to look at and maybe use if it does what you need? main...ne0rrmatrix:MauiOld:TextureViewIdea

Thanks and you are correct I missed that but that is not the issue. I have removed the xml completely (deleted it or removed the include on csproj) and it still won't build to Android without:

"Java.Lang.AbstractMethodError: 'abstract method "void androidx.media3.common.Player$Listener.onSurfaceSizeChanged(int, int)"'"

This is just doing the steps I said and no code changes at all. I am just trying to get a working demo project that references CommunityToolkit so I can modify Community Toolkit and see the changes in the demo project. But I don't know how to make that work. Irrespective of any code changes.

As noted, I am getting Handler doesn't exist errors if I don't add the manual handler association in mauibuild.cs and I am getting Java errors on the current construction of the PlayerView. So clearly something is wrong. This is again with no code changes.

Any idea? Did I do the right steps to make this work? Was I supposed to do something else?

I actually need to fix Plugin.Maui.Audio next so it is useful for me to know how this is supposed to work.

[ne0rrmatrix]

@jonmdev I will be posting a sample you can work from in about 5 min. Here is quick image of what I have done so far.

[ne0rrmatrix]

Here is latest changes. It is working but there is no API. You will need to add that if you want to create a PR for it. I hope this is what you were looking for. https://github.com/ne0rrmatrix/MauiOld/tree/TextureViewIdea If it does not do what you need please reply back.

[ne0rrmatrix]

The linked GitHub project has working sample. Build it. Goto view. Media element. Then load texture view page.

…

On Mon, Feb 17, 2025, 10:47 jonmdev ***@***.***> wrote: I put together a quick and dirty sample for you to look at and maybe use if it does what you need? main...ne0rrmatrix:MauiOld:TextureViewIdea <main...ne0rrmatrix:MauiOld:TextureViewIdea> Thanks and you are correct I missed that but that is not the issue. I have removed the xml completely (deleted it or removed the include on csproj) and it still wont' build to Android without: "*Java.Lang.AbstractMethodError:* 'abstract method "void androidx.media3.common.Player$Listener.onSurfaceSizeChanged(int, int)"'" This is just doing the steps I said and no code changes at all. I am just trying to get a working demo project that references CommunityToolkit so I can modify Community Toolkit and see the changes in the demo project. But I don't know how to make that work. Irrespective of any code changes. As noted, I am getting Handler doesn't exist errors if I don't add the manual handler association in mauibuild.cs and I am getting Java errors on the current construction of the PlayerView. So clearly something is wrong. This is again with no code changes. Any idea? Did I do the right steps to make this work? Was I supposed to do something else. — Reply to this email directly, view it on GitHub <#2526 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA7ZRN6XLDDDOCARH7APNV32QIVFNAVCNFSM6AAAAABXIU6MSSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRTHA4DAMBWGA> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: jonmdev]*jonmdev* left a comment (CommunityToolkit/Maui#2526) <#2526 (comment)> I put together a quick and dirty sample for you to look at and maybe use if it does what you need? main...ne0rrmatrix:MauiOld:TextureViewIdea <main...ne0rrmatrix:MauiOld:TextureViewIdea> Thanks and you are correct I missed that but that is not the issue. I have removed the xml completely (deleted it or removed the include on csproj) and it still wont' build to Android without: "*Java.Lang.AbstractMethodError:* 'abstract method "void androidx.media3.common.Player$Listener.onSurfaceSizeChanged(int, int)"'" This is just doing the steps I said and no code changes at all. I am just trying to get a working demo project that references CommunityToolkit so I can modify Community Toolkit and see the changes in the demo project. But I don't know how to make that work. Irrespective of any code changes. As noted, I am getting Handler doesn't exist errors if I don't add the manual handler association in mauibuild.cs and I am getting Java errors on the current construction of the PlayerView. So clearly something is wrong. This is again with no code changes. Any idea? Did I do the right steps to make this work? Was I supposed to do something else. — Reply to this email directly, view it on GitHub <#2526 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA7ZRN6XLDDDOCARH7APNV32QIVFNAVCNFSM6AAAAABXIU6MSSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRTHA4DAMBWGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ne0rrmatrix]

Here is a copy of your project updated to dotnet 9 and is now not crashing. https://github.com/ne0rrmatrix/MediaElementOverlapBug/tree/master

[jonmdev]

Thanks @ne0rrmatrix but as I said, I need to also do this with Plugin.Maui.Audio (as that is missing audio compression options which I already solved similarly ~8 months ago but similarly by just copying and pasting the code out rather than pushing an update). The project owners for that similarly asked me to update the project but I didn't know how (same problem as now).

So I really need to know what to do start to finish to do this. (Not the code fix itself, but the process of working this way.)

I have deleted everything, started again now from the beginning, and enumerated all my steps in detail. I hope you or @bijington can tell me perhaps what I am doing wrong.

(i) FORK & BRANCH COMMUNITY TOOLKIT

1. Go to https://github.com/CommunityToolkit/Maui and click Fork, name it "Maui-CommunityToolkit", click Create
2. In Github Desktop, clone this new fork to my hard drive from its page (https://github.com/jonmdev/Maui-CommunityToolkit)
3. Click "Current Branch" in Github Desktop UI and add new branch named "AddAndroidTextureView"
4. Get prompted "How are you going to use this project?" Select "To contribute to the parent project" and Continue
5. Publish Branch (with no changes made at this point)

(ii) ADD A BASIC TEST PROJECT TO PLAY WITH

1. Open samples/CommunityToolkit.Maui.Sample.sln on my system in VS2022.
2. Right click on the Samples folder and select Add > New Project, .NET Maui project, named "AddAndroidTextureViewTest"
3. Right click this and "Set as Startup Project".
4. Right click Dependencies for this project and select "Add Project Reference"
5. Get offered a list of all the Maui Community Toolkit packages I can tick, tick off CommunityToolkit.Maui, CommunityToolkit.Maui.Core, CommunityToolkit.Maui.MediaElement, CommunityToolkit.Maui.MediaElement.Analyzers, and CommunityToolkit.Maui.MediaElement.Analyzers.CodeFixes, press OK
6. This then provokes an error "Warning As Error: Detected package downgrade: Microsoft.Maui.Controls from 9.0.40 to 9.0.14" which I fix by editing the AddAndroidTextureViewTest.csproj to Include="Microsoft.Maui.Controls" Version="9.0.40" (make it version specific)
7. Can build at this point basic Maui project to Android emulator, so push Git update for this branch here to save progress.

(iii) ADD MEDIA ELEMENT TO TEST PROJECT

1. Go to mauibuild.cs of this test project and add .UseMauiCommunityToolkit().UseMauiCommunityToolkitMediaElement() to builder chain.
2. Go to app.xaml.cs (or anywhere and add MediaElement mediaElement = new() and add to hierarchy.
3. Get the following error now on trying to build to Android:

So again this is with no changes to any underlying code. I am just trying to set up a basic sandbox (default Maui test project) with a reference to my branched/forked CommunityToolkit so I can demo the MediaElement changes once I make them and it gives me this error.

Did something change in the implementation of MediaElement? Were my steps wrong above?

I have pushed an update so it is now in the exact state as above and I documented every step as I took it so nothing was missed. https://github.com/jonmdev/Maui-CommunityToolkit/tree/AddAndroidTextureView

Was I supposed to do something else instead? Thanks for any suggestion.

[ne0rrmatrix]

I am happy to help you if you want to create a PR and add the feature you have requested. I provided a fixed sample of you app that has been updated for dotnet 9.x. If you need help with Plugin.Maui.Audio I suggest going to that repo and starting a discussion with the author about it.

[jonmdev]

I am happy to help you if you want to create a PR and add the feature you have requested. I provided a fixed sample of you app that has been updated for dotnet 9.x. If you need help with Plugin.Maui.Audio I suggest going to that repo and starting a discussion with the author about it.

I am just trying to understand how to do this process. It is not specific for one package or another since I presume this requires the same workflow either way. (Point of reference @bijington was the one who suggested the compression options should be committed if someone is willing, so again same story again in both threads with my inability here.)

I listed all the steps I took to set up the forked repo and a demo project and yet I am getting an error from that with no changes to the underlying CommunityToolkit. Do you know if I followed all the steps correctly as listed or why my workflow is not working?

If I did something wrong, what are the correct steps to fork the project and add a sample to it with a reference that works? If you are not keen on discussing that it's fine of course (you owe me nothing obviously), but I cannot do anything meaningful in any greater sense with any of this unless I understand how to at least set up a basic configuration. @bijington can you perhaps explain?

[ne0rrmatrix]

You did nothing wrong. I was confused by what you were asking. You referenced this project, a different project, and I was unable to see what you needed help with or how to proceed.
Good news. I looked at your github fork and you did it right! If you take a look at my fixed sample of your app and apply the changes I made and move your code out of app.xaml.cs and into Mainpage.xaml.cs it should be fixed. I have not tested it directly but you should not be building pages and loading media element inside App.xaml.cs.

[jonmdev]

You did nothing wrong. I was confused by what you were asking. You referenced this project, a different project, and I was unable to see what you needed help with or how to proceed. Good news. I looked at your github fork and you did it right! If you take a look at my fixed sample of your app and apply the changes I made and move your code out of app.xaml.cs and into Mainpage.xaml.cs it should be fixed. I have not tested it directly but you should not be building pages and loading media element inside App.xaml.cs.

Thank you! That is very helpful to know I at least set it up correctly.

So it appears then perhaps I have unveiled a new bug. I tested and the bug is confirmed from everything I can see. I posted a bug report on it accordingly: #2529

I will wait for any ideas or suggestions on fixing that before proceeding with this as I cannot work in C# with MediaElement unless I use MediaElement mediaElement = new(); absLayout.Add(mediaElement); and currently this approach no longer seems to work in Android.

Thanks again for looking at things and at least letting me know my method of setting up the project was correct.

[bijington]

@jonmdev i am more than happy to assist where I can. It sounds like your approach is a good one and the steps look right. You may have been able to avoid creating a sample project and instead use the sample that comes with the toolkit codebase - it is a lot more complex than a simple application though so just use whatever works best for you.

As for other steps I'd be more than happy to walk you through bits on a call or via email or on here, again whatever works best for you.

I really appreciate you putting in the effort to making the toolkit better. I'd also be happy to help out with the audio plugin

[jonmdev]

SOLUTION

Here is my solution. I will explain in words what I did so it is easy to review if that helps.

A lot of the changes you will see there are due to the default Maui project I added for testing as samples/AddAndroidTextureViewTest which obviously does not need to go into the final project. We can presumably delete that rather than merge it.

I also manually added the fix here to MediaManager.android.cs as otherwise you cannot work with anything in Android currently. So that is redundant but no harm (same thing).

Here are the steps of the solution.

Add to csproj to Include textureview.xml

<!--include the textureview.xml needed for creating textureview in android-->
  <ItemGroup Condition="$(TargetFramework.Contains('-android'))">
    <Folder Include="Platforms\Android\Resources\layout\" />
    <AndroidResource Include="Platforms\Android\Resources\layout\textureview.xml" />
  </ItemGroup>

Add MediaElementOptions to MediaElementOptions.cs:

/// <summary>
/// Construction options for MediaElement, for example to create an Android SurfaceView or TextureView
/// </summary>
public class MediaElementOptions 
{
	/// <summary>
	/// Set Android View type for MediaElement as SurfaceView or TextureView on construction
	/// </summary>
	public AndroidViewType AndroidViewType = AndroidViewType.SurfaceView;
}

/// <summary>
/// Enum for Android view type, whether to make MediaElement as SurfaceView or TextureView
/// </summary>
public enum AndroidViewType {
	/// <summary>
	/// Create Android MediaElement as SurfaceView
	/// </summary>
	SurfaceView,
	/// <summary>
	/// Create Android MediaElement as TextureView
	/// </summary>
	TextureView
}

I put this all in one file in Views as I figure it will be easiest to read for users that way especially as new options might need to be added. If many options are ever later added, it could be split to multiple files or its own folder.

Add MediaElement Constructor to Take the Options

/// <summary>
/// New Constructor with Options
///</summary>
public MediaElement(MediaElementOptions? mediaElementOptions = null) 
{
	// Assign default options if none is set as argument
	if (mediaElementOptions == null) { mediaElementOptions = new(); }

	// Save to the object for handler to access during CreatePlatformView
	this.MediaElementOptions = mediaElementOptions;
}

Based on the current more "Rust" no-null style approach of C#, and given virtually zero data cost to any of this, I thought the simplest solution was to be sure MediaElementOptions and everything inside it is non-null once you go past this constructor. MediaElementOptions will initialize to the defaults set inside that class past this point if set in as null. This is safe and easy. It also allows users to safely read the construction options used to build after building whether they purposely set them or not.

In MediaElementHandler Get the Options from MediaElement and Use in MediaManager.CreatePlaformView(options)

protected override MauiMediaElement CreatePlatformView()
{
	mediaManager ??= new(MauiContext ?? throw new InvalidOperationException($"{nameof(MauiContext)} cannot be null"),
							VirtualView,
							Dispatcher.GetForCurrentThread() ?? throw new InvalidOperationException($"{nameof(IDispatcher)} cannot be null"));

	// get the options from the MediaElement constructor
	MediaElementOptions options = VirtualView.MediaElementOptions;

	var (_, playerView) = mediaManager.CreatePlatformView(options);
	return new(Context, playerView);
}

I modified all the MediaManager.CreatePlatformView functions to take a MediaElementOptions object so this will allow cross-platform construction customization if ever needed with no further changes to the general infrastructure.

Use the Construction Options in Android's MediaManager.CreatePlatformView to make a TextureView (or SurfaceView):

	public (PlatformMediaElement platformView, PlayerView PlayerView) CreatePlatformView(MediaElementOptions mediaElementOptions)
	{
		Player = new ExoPlayerBuilder(MauiContext.Context).Build() ?? throw new InvalidOperationException("Player cannot be null");
		Player.AddListener(this);

	
		// Create SurfaceView or TextureView
		if (mediaElementOptions.AndroidViewType == AndroidViewType.SurfaceView) {

			// Create SurfaceView
			PlayerView = new PlayerView(MauiContext.Context) {
				Player = Player,
				UseController = false,
				ControllerAutoShow = false,
				LayoutParameters = new RelativeLayout.LayoutParams(ViewGroup.LayoutParams.MatchParent, ViewGroup.LayoutParams.MatchParent)
			};
		}
		else {

			// Create TextureView
			Context context = MauiContext.Context!; 
			Android.Content.Res.Resources resources = context.Resources!; 
			System.Xml.XmlReader xmlResource = resources.GetXml(Microsoft.Maui.Resource.Layout.textureview);
			xmlResource.Read();
			Android.Util.IAttributeSet attributes = Android.Util.Xml.AsAttributeSet(xmlResource)!; 

			PlayerView = new PlayerView(MauiContext.Context, attributes) {
				Player = Player,
				UseController = false,
				ControllerAutoShow = false,
				LayoutParameters = new RelativeLayout.LayoutParams(ViewGroup.LayoutParams.MatchParent, ViewGroup.LayoutParams.MatchParent)
			};
		}

Make PlayerView Accessible in MauiMediaElement so You can Read it After Build

The PlayerView is stored in MauiMediaElement (which is the Platform View), such that in Android we get:

var platformView = ((CommunityToolkit.Maui.Core.Views.MauiMediaElement)mediaElement.ToPlatform(mediaElement.Handler!.MauiContext!));	

However, you could not previously even read the PlayerView from this object without Reflection, which I have thought is counterproductive in case people want to apply further customizations to it at their discretion or even read it just to assess the built view type in this case. I think this is unnecessarily paranoid/paternalistic/unhelpful. I could not even assess whether my code was working directly from my sample project in this system due to this. So I made a minor switch to change it to:

public PlayerView PlayerView { get; }

I think this is still safe and reasonable. It may allow users the range to further customize the PlayerView without causing any major trouble if other things come up (based on their own expertise). This then allowed me to test and debug out the type with the following code in my sample project App.xaml.cs:

	public MyTestPage() {

		//======================================
		//SET THE ANDROID VIEW TYPE TO CREATE
		//======================================
		MediaElementOptions options = new() { AndroidViewType = AndroidViewType.TextureView };
		//MediaElementOptions options = new() { AndroidViewType = AndroidViewType.SurfaceView };

		MediaElement mediaElement = new(options);
		//MediaElement mediaElement = new(); // default is SurfaceView

		mediaElement.HandlerChanged += delegate {

#if ANDROID
			//============================================
			//TEST IF CREATED CORRECT ANDROID VIEW TYPE
			//============================================
			var platformView = ((CommunityToolkit.Maui.Core.Views.MauiMediaElement)mediaElement.ToPlatform(mediaElement.Handler!.MauiContext!));
			var viewType = platformView.PlayerView.VideoSurfaceView?.GetType();
			Debug.WriteLine("ANDROID VIEW TYPE: " + viewType?.ToString() ?? "");

			//=== outputs either:
			// ANDROID VIEW TYPE: Android.Views.TextureView
			// ANDROID VIEW TYPE: Android.Views.SurfaceView
#endif
		};

You can comment in or out lines in this test project to set the MediaElementOptions to SurfaceView or TextureView. Or set MediaElementOptions as null or don't supply it to the constructor.

All options work fine. Default remains SurfaceView in Android. But if you set it to TextureView you get a TextureView. Building in Android will debug out the Android view type as one of the two examples listed above from the test project.

Logic of Approach

I wanted to approach this in the simplest and easiest way that I thought would also be expandable for other construction parameterization in all platforms if needed.

I think this accomplishes that goal with virtually no computational cost and very low user complexity.

Personally this is how I would want to use MediaElement. I can't think of anything easier or more intuitive.

C# Usage

MediaElement mediaElement = new(new MediaElementOptions(){ AndroidViewType = AndroidViewType.TextureView });

or:

var options = new MediaElementOptions(){ AndroidViewType = AndroidViewType.TextureView };
MediaElement mediaElement = new(options);

or the original default function which still works of course:

MediaElement mediaElement = new();

Xaml Usage

As noted above, I do not personally work in xaml, but users could easily create a derived class of MediaElement like MediaElementTextureView which takes no arguments but in its constructor, build itself with the new TextureView constructor method. Then they can use that in their xaml the same as usual (only now it will build a TextureView). That is still easy for them.

public class MediaElementTextureView : MediaElement
{
    // Parameterless constructor for XAML usage
    public MediaElementTextureView()
        : base(new MediaElementOptions { AndroidViewType = AndroidViewType.TextureView }) { }
}

Review?

What do you think? What do we do next either way?

I presume if merging at some point, I would first delete the sample project and the redundant changes from the other fix will get matched as no change if they get merged first?

Thanks for teaching me how to do this up to this point. Hopefully we can get something merged in that will work as well and I will be able to switch back into using this long term. 🙂👍

[bijington]

Thanks for the detailed breakdown, usually I would say it's easier to view in a PR but this is really helpful!

The one issue I can spot on first glance is the approach of adding a parameter to the constructor for MediaElement, this would not be usable in XAML. I think we could follow the options pattern like is already done for converters, etc. I do like the approach for options though so I like the idea. In fact I had a PR open that tried to add in options for MediaElement - I see this as just setting the defaults and developers can still override on a per instance basis

[jonmdev]

Thanks for the detailed breakdown, usually I would say it's easier to view in a PR but this is really helpful!

The one issue I can spot on first glance is the approach of adding a parameter to the constructor for MediaElement, this would not be usable in XAML. I think we could follow the options pattern like is already done for converters, etc. I do like the approach for options though so I like the idea. In fact I had a PR open that tried to add in options for MediaElement - I see this as just setting the defaults and developers can still override on a per instance basis

Glad to hear it. I have thought about the same things you mention as well. As I said in my post, this can still be used in xaml by deriving the class:

public class MediaElementTextureView : MediaElement {
    public MediaElementTextureView()
        : base(new MediaElementOptions { AndroidViewType = AndroidViewType.TextureView }) { }
}

This MediaElementTextureView a user can make with 4 lines of code as above can then be used in xaml anywhere in code (I believe). That is a pretty simple solution in my opinion any user can do easily. That example can be added to documentation if needed to illustrate for people how to do it. Adding such 4 lines of code is much simpler than many Maui system configuration tasks certainly. Have you seen how you configure the new HybridWebView json system? 😂

I am not aware of any other good option for this other than to set the options as a parameter. I believe we have no other opportunity to certainly access the underlying ExoPlayer PlayerView Android constructor otherwise before it is too late.

Once we do MediaElement mediaElement = new(); the Handler system takes over and constructs it at whatever speed it can get it done, right? So except as an argument we have no practical way otherwise of setting anything in that is guaranteed to be present on construction. I don't think it should be hard for people to create a 4-line derived class like above if they want TextureView functions in xaml.

The only other way I can think of it could be done would be to let it build as it is (no constructor parameter), then run a function to change the view type after building. But this is then highly inefficient and dangerous, as we will have to build a SurfaceView by default and then discard it from the Android hierarchy to create a new TextureView to replace it afterwards and it is just asking for trouble in my opinion to do it that way. It is way, way, way more complicated for a considerably slower and less efficient method.

We will have to reset all the parameters over to the new TextureView as we hotswap it out from the default SurfaceView and this is just slowing everything down and dangerous if something goes wrong. If the SurfaceView starts playing and then we need to transfer the playing state this could create major glitches and weird undefined behavior. There is no practical reason people would want to switch the View type after building either - you would want one or the other for a given use case, so I believe this would be highly overengineering a simple problem with too much danger and no obvious reward.

There may not be a perfect solution but I think the current proposal is great in my opinion (fits everything I need certainly but perhaps I am biased 🙂) and certainly it is at least better than the current circumstance where we have no TextureView at all.

This method would provide the best performance and safest most efficient construction, because we are guaranteed to only construct the Android view once, which is also important in the real world. The method proposed is safe and guaranteed to work every time without new bugs.

You know more about these things than I do so if you know of another way of safely getting the MediaElementOptions object into the Handler before it constructs the platform view (and only before so users don't try to set it after it builds which again is a poor idea in my opinion that almost no one would ever want), then I am open minded.

Otherwise I think this is the best approach from all the time I have thought about it and everything I know, which is why I went this route. I think the safety and speed of the method are very important, especially on mobile devices and this is maximally safe and efficient.

Thanks again and for any further thoughts.

[bijington]

I have a thought on a pattern we can use to achieve this without having to supply it as a parameter in the constructor but first I would like to take a step back and ask whether there is ever a time a developer would not want to use TextureView? If the answer is no how about we don't make it configurable but just make it the implementation?

[ne0rrmatrix]

SurfaceView is the default for exoplayer. Switching to texture view is an option that available. It is not advised to use it unless you have a specific need for it. https://developer.android.com/media/media3/ui/playerview#surfacetype

[ne0rrmatrix]

The surface_type attribute of PlayerView lets you set the type of surface used for video playback. Besides the values spherical_gl_surface_view (which is a special value for spherical video playback) and video_decoder_gl_surface_view (which is for video rendering using extension renderers), the allowed values are surface_view, texture_view and none. If the view is for audio playback only, none should be used to avoid having to create a surface because doing so can be expensive.

If the view is for regular video playback then surface_view or texture_view should be used. SurfaceView has a number of benefits over TextureView for video playback:

Significantly lower power consumption on many devices.
More accurate frame timing, resulting in smoother video playback.
Support for higher quality HDR video output on capable devices.
Support for secure output when playing DRM-protected content.
The ability to render video content at the full resolution of the display on Android TV devices that upscale the UI layer.
SurfaceView should therefore be preferred over TextureView where possible. TextureView should be used only if SurfaceView does not meet your needs. One example is where smooth animations or scrolling of the video surface is required prior to Android 7.0 (API level 24), as described in the following notes. For this case, it's preferable to use TextureView only when SDK_INT is less than 24 (Android 7.0) and SurfaceView otherwise.

Qouted from google site: https://developer.android.com/media/media3/ui/playerview#surfacetype

[jonmdev]

I have a thought on a pattern we can use to achieve this without having to supply it as a parameter in the constructor but first I would like to take a step back and ask whether there is ever a time a developer would not want to use TextureView? If the answer is no how about we don't make it configurable but just make it the implementation?

Yeah as @ne0rrmatrix said SurfaceView is the default. This is because it is more efficient. It is more of a direct rendering to the screen, which is why it doesn't obey normal layout rules and overlays. I have read people describe it as being a "punched hole" through the screen in the rendering system.

So if you just want to make an efficient full screen video player (like for watching 2 hour Hollywood movies), you will typically want SurfaceView. If you want something that integrates into a modern Maui layout, with overlapping elements or rounded corners, etc., you will want TextureView.

I am open to any ideas for how it could work as long as we have both options and the method does not necessitate any unsafe or unnecessary inefficiencies like building then destroying the entire default view before remaking the correct desired view.

As noted, 99.99999% of the time you will need one or the other. There is no practical reason to switch once it is made, and if you are going to switch in some rare edge case, I think you are better off in my opinion discarding the entire MediaElement since the Exoplayer (whether surfaceview or textureview) essentially IS the MediaElement for Android than trying to rewrite the entire guts to handle live swapping out the core without causing catastrophe.

Thanks. Almost done adding AAC audio compression to Plugin.Maui.Audio also. I'll post that likely tomorrow.

[bijington]

Thank you both for the detailed responses. I think rather than requiring developers to need to subclass MediaElement we could achieve it as follows:

Add options

Enable the UseCommunityToolkitMediaElement extension method to take an options parameter of your type MediaElementOptions, this could set a static property somewhere to store the value passed in. I much prefer the Action approach that we followed with Plugin.Maui.Audio as it reads nicer when a developer writes it - I am not set on this though as I appreciate it might conflict with the current implementation in this toolkit.

It could be used like:

builder.UseCommunityToolkitMediaElement(options =>
{
#if ANDROID
    options.ViewType = AndroidViewType.TextureView;
#endif
});

Internally this could set something like MediaElement.DefaultViewType which is an internal static property.

Then we just use this when a new MediaElement is created.

Failing that I wonder if there is an approach we could use through the handler architecture although as you suggested it is likely too late for this.

[ne0rrmatrix]

Here is an example of how to use options in Media Element based on how we do it for other things. https://github.com/ne0rrmatrix/MauiOld/tree/MediaElementOptions

This example was made as a possible solution to something else but should demonstrate how it can be done.

[ne0rrmatrix]

Here is a complete example of texture view being implemented if you are looking for an example. https://github.com/ne0rrmatrix/MauiOld/tree/TextureViewIdea

It includes adding a sample page for texture view. It is supported out of the box for windows and ios. It just works. It uses an options class and it works like other option classes we currently uses. It also integrates with current methods and classes.

It is based upon @jonmdev code.

[jonmdev]

Thank you both for the detailed responses. I think rather than requiring developers to need to subclass MediaElement we could achieve it as follows:

Add options

Enable the UseCommunityToolkitMediaElement extension method to take an options parameter of your type MediaElementOptions, this could set a static property somewhere to store the value passed in. I much prefer the Action approach that we followed with Plugin.Maui.Audio as it reads nicer when a developer writes it - I am not set on this though as I appreciate it might conflict with the current implementation in this toolkit.

It could be used like:

builder.UseCommunityToolkitMediaElement(options =>
{
#if ANDROID
options.ViewType = AndroidViewType.TextureView;
#endif
});

Internally this could set something like MediaElement.DefaultViewType which is an internal static property.

Then we just use this when a new MediaElement is created.

Failing that I wonder if there is an approach we could use through the handler architecture although as you suggested it is likely too late for this.

Personally, this would be less ideal in my opinion as it would then force users to always use only one type of Android View for all Android MediaElement's throughout your project.

Whereas while it may not be considered beautiful to derive from MediaElement to customize it, we still preserve both methods safely for all users and they can use both throughout their project with zero bugs or errors as well in doing so.

I also think 4 lines of deriving a C# class off in a small file somewhere is significantly less intrusive than adding potentially many lines of "options" into the builder chain of MauiProgram.cs but this is perhaps subjective. My MauiProgram.cs is already a mess of compiler flags and other configuration information. 😂

My perspective in general on this subject is that "perfect" does not need to be the enemy of "good". Currently for the entire lifetime of this MediaElement project there has been no TextureView support at all. This method safely and efficiently adds it with full backwards compatibility. No one's current projects will be broken in any way by the addition. It just adds an extra option which is easy and safe to use. It allows full flexibility. And it is absolutely safe and fast. To me that's a good step forward and progress is good.

Btw, I just finished Plugin.Maui.Audio AAC audio compression and posted it here. But we can talk about that over there. 🙂

[jonmdev]

It just occurs to me @bijington , perhaps we could even do both methods?

Why not?

1) GLOBAL SETTING

We could have your method as one option to set default options globally for the project:

builder.UseCommunityToolkitMediaElement(options => {
    options.ViewType = AndroidViewType.TextureView;
});

(No compiler flags are needed as setting this doesn't hurt non-Android builds and this code is cross platform safe.)

This would set the global default "options" used on construction of all MediaElements after that in the project

2) INDIVIDUAL OVERRIDE

Then in addition, we can also let people use my method:

var options = new MediaElementOptions(){ AndroidViewType = AndroidViewType.TextureView };
MediaElement mediaElement = new(options);

or for xaml:

public class MediaElementTextureView : MediaElement
{
    public MediaElementTextureView()
        : base(new MediaElementOptions { AndroidViewType = AndroidViewType.TextureView }) { }
}

LOGIC

The logic would be that on MediaElement construction the constructor is changed to:

public MediaElement(MediaElementOptions? mediaElementOptions = null) 
{

	// SOLVE DEFAULT IF NO ARGUMENT SET (OTHERWISE USE ARGUMENT AS OVERRIDE)
	if (mediaElementOptions == null) { 

    	     // (1) CHECK FOR STATIC OR GLOBAL "OPTIONS" SET IN ON BUILDER

    	     // (2) IF NOTHING IS FOUND CREATE NEW DEFAULT
             mediaElementOptions = new(); 
        }

	// USE SOLVED RESULT
	this.MediaElementOptions = mediaElementOptions;
}

Thus the order of operations for solving the settings is:

• If nothing set into MediaElement constructor (null argument), check for the "Global" option set (stored in a static field somewhere presumably by builder chain) to use.
• If nothing set into MediaElement constructor and nothing set to "Global" option on builder, use default new() options object.
• If options manually set by user into MediaElement constructor, use that as override and take that instead for construction.

RESULT

Thus with this combined approach you can allow many possibilities:

• XAML users who do not wish to derive classes can set a global builder option.
• C# users who only have one view type needed can also use a global builder option to set it for the whole project.
• C# and XAML users can both also still now override and individually set this per MediaElement with multiple ways to work (derive for XAML, simple code for C#).

Not bad I think? And still maximally safe and fast.
🙂🍻

I will add that now and we can see.

[jonmdev]

UPDATE

That was incredibly simple to add. Both methods for setting the MediaElementOptions are now working and I have added the code here. To summarize, here are the new changes.

Add MediaElementOptions Argument to UseMauiCommunityToolkitMediaElement

	public static MauiAppBuilder UseMauiCommunityToolkitMediaElement(this MauiAppBuilder builder, MediaElementOptions? mediaElementOptions = null)
	{
		builder.ConfigureMauiHandlers(h =>
		{
			h.AddHandler<MediaElement, MediaElementHandler>();
		});

		// Assign default MediaElement options if not null
		if (mediaElementOptions != null) {
			MediaElementBuilderOptions.MediaElementOptions = mediaElementOptions;
		}

#if ANDROID
		builder.Services.AddSingleton<Media.Services.MediaControlsService>();
#endif
		return builder;
	}

Store Default Options into Simple Static Class

static class MediaElementBuilderOptions
{
	// Default options set by UseCommunityToolkitMediaElement(options) 
	public static MediaElementOptions MediaElementOptions = new(); //never null
}

I have ensured the stored options are never null with the above so we can always safely refer to this for the default. This should be simple and safe and it is quick and easy for the task.

Check Global Default MediaElementOptions if No Argument Given to MediaElement Constructor

The new MediaElement constructor is now then to integrate all this:

public MediaElement(MediaElementOptions? mediaElementOptions = null) 
{
	// Assign default options (set by builder or options constructor) if null is set as argument
	if (mediaElementOptions == null) { 
		mediaElementOptions = MediaElementBuilderOptions.MediaElementOptions; 
	}

	// Save to the object for handler to access during CreatePlatformView
	this.MediaElementOptions = mediaElementOptions;
}

USER EXAMPLES:

(1) Global Setting By Builder:

var builder = MauiApp.CreateBuilder();
builder
	.UseMauiApp<App>()
	.UseMauiCommunityToolkit()
	.UseMauiCommunityToolkitMediaElement(new MediaElementOptions() {
		AndroidViewType = AndroidViewType.TextureView
	})

(2) C# Override Per Object:

MediaElementOptions options = new() { AndroidViewType = AndroidViewType.TextureView };
MediaElement mediaElement = new(options);

(3) XAML or C# Override Per Object:

public class MediaElementTextureView : MediaElement
{
    public MediaElementTextureView()
        : base(new MediaElementOptions { AndroidViewType = AndroidViewType.TextureView }) { }
}

I think that's pretty good, very safe, very fast, very easy and lots of choices. I like that a lot and I think that addition makes it much better.

What do you think?

[bijington]

I think the approach sounds good. Do you think you could open a PR so that it will be easier to review and potentially merge?

[jonmdev]

I think the approach sounds good. Do you think you could open a PR so that it will be easier to review and potentially merge?

Cool! Done I think.

#2540