Feature Request: Caching of feed to prevent re-run when no changes are are available

[EternalBlack]

Description

We got a pretty big setup currently with about ~100 Feeds that all should be rerun hourly (or even quicker, if possible). I already made a script that triggers all feeds to re-import and update the entries. Even though there are no changes inside the feed, feed-me still fully works through all of them (In my case taking about 30-45 Min for all ~100 Feeds).
Would be awesome if feed me kind of caches the feeds and compares the locally cached XML with the one at the URL (by content, should be the most reliable option I guess). I assume this would speed up the import phase tremendously as it currently takes up kind of much time.

Steps to reproduce

1. Start all feeds.
2. Wait for them to finish.
3. Wait reeeeeeal long for them to finish even though nothing changed. ;D

Additional info

• Plugin version: 3.1.16 Pro
• Craft version: 3.1.19 Pro

[engram-design]

There are mechanisms in Craft to check existing elements' content with what the resulting imported one may be, but this might be another solution. However, I have seen many feeds that include some unique content like a timestamp, which changes on every access, so in that scenario, it wouldn't be helpful.

Adding it to the list!

[EternalBlack]

I'm currently pretty close to getting "caching" (i.e. preventing a re-run if no changes are available inside the feed-url) to work already. Currently struggling with the fact that EVENT_AFTER_FETCH_FEEDs $event doesn't provide any other info than the $url and $response. This already actually works for feeds that don't share the same URL. But if one feed imports for entries of section1 and another imports for entries of sectionXYZ but both using the same URL to import will prevent the last one from triggering correctly. So it also might be cool to have the options that a single feed(URL) provides multiple actions (we currently need to always create 2 feeds every time as we already have this kind of problem where we need to split up the data from the feedURL in two different sections, but thats another story). Back to topic:

I currently use

Event::on(DataTypes::class, DataTypes::EVENT_AFTER_FETCH_FEED, function (FeedDataEvent $event) {
	$folderName = hash('md5', $event->url);
	$path = Craft::getAlias('@storage/feedcache/' . $folderName);
	if (file_exists($path)) {
		if (array_key_exists('data', $event->response)) {
			$filename = basename($event->url);
			$file = $path . '/' . $filename;

			//temp save content to file
			!is_dir($path) && !mkdir($path, 0777, true) && !is_dir($path);
			file_put_contents($file . 'new', $event->response['data']);
			// compare files

			if ($this->files_identical($file, $file . 'new')) {
				//abort here!
				$event->response = null; // Don't if this is the way to do it...

			}
		}
	}
});

Event::on(Process::class, Process::EVENT_BEFORE_PROCESS_FEED, function (FeedProcessEvent $event) {
	$feed = $event->feed;
	if (array_key_exists('feedUrl', $feed)) {
		$feedURL = $feed['feedUrl'];
		$folderName = hash('md5', $feedURL);

		$path = Craft::getAlias('@storage/feedcache/' . $folderName);
		!is_dir($path) && !mkdir($path, 0777, true) && !is_dir($path);

		$filename = basename($feedURL);
		copy($feedURL, $path . '/' . $filename);
	}
});
// files_identical -> https://www.php.net/manual/en/function.md5-file.php#94494

Having 2 feeds (ID:300 and ID:301) with both the same feedUrl that are run sequentially will result in the one that is triggered later to just abort without doing any action as the feed was just fetched and the file that was copied is still the same even though the job of that feed is a different one than the first.
Having EVENT_AFTER_FETCH_FEEDs $event provide the id of the feed, one could use that to more precisely identify the file via name (just append the id to the filename). Even though this would also result in having to download the "feed-url-file" over and over for each feed that uses the same URL even though it's actually only needed once. But I think that would be ok for now and might be optimized later (as suggest earlier, this could be fixed by allowing to have multiple actions per feed URL instead of treating all feeds as a single object for itself).