How will we handle running code and storing data? #67
Comments
|
well, the same question comes up as always. who's paying? do you have $400 laying around or a job that pays over $180/month? because i don't think anybody here will want to pay that when an even better and cheaper idea would be to grab an old office computer (like a dell or smthn, doesn't really matter) off ebay and throw a few HDDs in there. that way, we'd be able to simply scale up the storage by buying some more cheap hard drives, and we'd also have the 250gb-1tb hard drive that comes with it. you're thinking way overkill for this small project (we prob wouldn't even need active cooling if we decide to go for a pi for, and we definitely wouldn't need ssds), and you aren't even putting yourself forward as the one who's gonna pay. |
i have an old dell computer |
|
Maybe use the free tier of MongoDB Atlas? |
aws lambda is free for compute. |
|
Cloud sounds expensive. I could hack something up in a few hours on my Pi and maybe give access to y'all through ssh, however. I'd need to find storage but I do have quite a bit of money I've been saving up, so we could get a reasonable storage configuration and a few performance upgrades for my Pi. Keep in mind that I do not want to buy a whole new Pi lol, so I'll just make do with my Pi 4B (which is still powerful enough to drive two 4K displays at 60fps, mind you). |
|
What is storage needed for if this is tapping in to the Scratch API? (I don't mean for Voyager, given that's now a separate project) |
I agree that the amount of storage is 50 magnitudes of overkill. You do need storage to be able to run the SQLite database, but that still shouldn't be that much of a problem. |
the pi 4b could easily do it. no need for nas thing i said.
Voyager is still part of SnarpleDev i think. Also, the reason i said that most likely inaccurate number is beacuse there are about 6.5 MILLION posts on scratch. |
|
We don't need to put all the 6.5 million posts on caché... |
not on cache... on some hdd or ssd |
|
UPDATE: |
nowhere near all of them are shared, and we have almost no reason to cache projects since there's an actual api |
oh right i forgot about that |
|
closing this because @redstone-dev pi might work |
I was actually thinking that lambda would be a great idea, but it's provably easier to run it on my pi since I recently deactivated my AWS account since I don't use it anymore :P |
Yeah, AWS is also planned for Voyager and that other thing we were talking about which I won't mention here |
|
might actually just run an instance of snazzle prod on my pi 4b, which is also where i've been developing it. would take care of this issue. as for voyager, i'd need more powerful hardware than just a single pi, so i'll have to either a) dig up an old PC and upgrade it, or b) get a pi 5. i'll probably try and get a pi 5 for this, along with 1tb of storage. also, speaking of voyager, how is that going? |
Great! I've successfully set it up to scrape all the first pages of all topics of all categories. I will eventually implement something to actually scrape all pages. |
nice, looking forward to using it for normal snazzle, and once we get the kinks worked out, snazzle prod :D |
Fenixio is actually the guy for Voyager's creation, while I'll just be doing the fun part of creating program structures, flowcharts, documentation and maintaining the damn thing by mergin' them dependabot PR's. Pretty easy stuff in comparison to making a program which scrapes literally a CENTIMILLION posts! |
Haha, but those are also important things! Without docs, organization or structure this whole project would spontaneously combust. |
|
We could deploy Snazzle on Vercel. It has a good free tier and supports flask. It also has git integration as far as I know. |
I remember Redstone denying Vercel as an option to hosting Snazzle, but I may remember wrong. It's up to Juniper for the final decision. |
Well obviously we aren't gonna do this stuff on our daily drived computers so these are my proposed two options:
Local: We make a NAS (Network-attached Storage) using a Raspberry Pi 5 (also runs code). This is the entire detailed thing for local.
The things we need: A Pi 5, A Penta SATA hat for the Pi 5 by Radxa some SSD's from either Crucial or Samsung and a active heatsink for the Pi 5.
Specifications: A 2.4 GHz quad-core Arm CPU, with cryptography extensions. 800 MHz VideoCore VII GPU for anything we might want to do. 8GB of LPDDR4X RAM. 2TB of SSD storage and 1 gigabit ethernet with plenty of I/O.
Procedure: First buy the stuff we need. Then, put the flex connector of the SATA hat into the PCIe slot. Then, put the other side into the SATA hat's port. Attach the active cooler of the Pi 5 and then attach the SATA hat on top of the Pi. Attach the SSD's to the SATA slots and we are good to go with assembly.
How each SSD is used: 2 SSD's store the English part of the forums we browse. The 3rd SSD will store everything else than the forums. And the 4th SSD stores the largest category on Scratch, Scratch around the world accounting for 99.7% of posts on the Scratch Forums. Currently no data backup, so put UPS for Pi...A
Cost: The price of the main rack costs about 150$ If you include SSD's then it's about 350$. Although costs to maintain is near 0.
Cloud: We can make this in the Cloud using Google Cloud because Google. Details below.
The things we need: Google Cloud Storage and Cloud Run
Specifications: God knows but we know we have 4TB of storage
Procedure: Buy Google Cloud Storage plan on free tier. Then start paying.
Cost: 173$ PER MONTH
so now @redstone-dev choice (also all of this storage issue idk how to do. this is just me being me and doing random stuff.)
The text was updated successfully, but these errors were encountered: