Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First Draft of Implementing PCFS #20

Merged
merged 12 commits into from
Jan 14, 2025
Merged

Conversation

Mythra
Copy link
Contributor

@Mythra Mythra commented Jan 14, 2025

This PR is absolutely massive, and I am sorry about that. PCFS requires so much work to get done (and even this still is missing a lot)! I probably should've split this up more, but would've lead to lot of churn as I couldn't really test any of this stuff without all the parts, and I wanted to keep the repo in a relatively build-able state with the same drawbacks that already existed.

So where is this at, and what's still missing?

  • You can now boot using PCFS IF:

    • You are booting up in the "default" state of a cat-dev, (e.g. using PCFS, with SATA enabled)
    • You already have a Cafe SDK checkout, and if copying from a case-insensitive to case-sensitive (e.g. windows to linux) you have manually validated all but title ids have correct capitilization. (You might also need to copy your temp directory depending on the state of other files, if your username changes between the two machines).
    • Your Cafe SDK directory is fully setup, and we are not required to create any files (e.g. you copy the files from a running host, before it shuts down, as FSEmul.exe will remove files we currently need as we can not generate them).
    • You can sometimes run into a timing issue, where the CAT-DEV will close a file, then query it's file info, and expect it to still be open, and this file info will be required for booting COS. I ahven't quite dug into this as it doesn't happen all the time, and sleeping on network latency just makes it worse. So I think we'll need to modify host filesystem to "wait" before closing files to more accurately emulate a shitty windows 7 pc.
  • What's still missing for PCFS?

    • not all PCFS commands are implemented, just the ones you will most likely need in order to boot the device.
    • Handling resolving of network mounts over PCFS with %NETWORK/, cross-os network mounts aren't really a thing, also there is likely to be more than one network mount, but PCFS doesn't support that. So we should figure out a good way to specify which network mount(s) you want to use, and how to handle that resolution.
    • PCFS SATA without support for FFIO & CSR turned on.
    • PCFS non SATA & non SDIO.
    • mounting in arbitrary other paths.
    • automated dumping.
    • COS Shell access through boot.
    • official executable reimplementations (these do some wild message bassing between each other which will take a much longer time than PCFS as a whole, named pipes are not easy to capture, debug, and follow through).
    • ATAPI is effectively an incredibly tiny stub, this is okay as that's all that's needed, but lots of ATAPI research is still needed.
    • Ownership takeover on a modern cat-dev bridge that supports takeover.

What Other Things Are In This PR?

  • Updated to the latest version of rust, and latest version of rust is required to build now. rustup update time if you want to build your own.
  • Reworked error code architecture, allowing finer grained errors in many cases, while simplifying so much of the error handling code.
  • Fixed massive issues with CLI handling in bridgectl, where it was not picking up correct arguments.
  • Fixed booting legacy cat-dev's with/without PCFS: Cat-Dev's with older MION FW fail to boot even without PCFS #18
  • Fixed bug in packages, where we were packaging binaries that tried to require openssl, rather than just libc/libm/libgcc.
  • Allow connecting to DEBUG_OUT for serial logs even if you don't have a serial port path connected.
  • Fix status page url we were fetching from to actually get proper setup arguments.

while i was working on this i also managed to implement the parsing
of Mion FW files. this requires a rebase, and a temporary checkin.
part of validating PCFS will be validating it across many MION FW
versions (thanks to crediar i was able to get quite a few more versions
to test with from SDKs they happened to have lying around). this adds
in commands to decrypt those firmwares, so i can actually figure out
what's going on with them.

it also adds a subcommand to quickly dump the fw out of running memory
incase someone new gets a cat-dev i wanna have a quick way to get their
current fw out of memory, without telling them they need to CTRL-C
early.

also small nit, fixed the help page rendering for the MION subcommand
as before it wasn't actually interacting correctly.
- this fixes arguments that _may_ not be a bridge not actually being
  handled in the correct order in `target_bridge`, it would always
  take the default over the positional argument.
- boot now can boot mion fw's that don't have access to `power_on_v2`.
- you can now get the targeted bridge mac easily without duplicate
  lookups when messing with `bridgectl`.
- `boot.rs` was becoming too long, so split it into it's own series
  of modules, can't say they'll always live this way, but they're
  better than what it was.
- fix status uri for getting mion status.
- build in route for ejecting discs on legacy MIONs
- build in route for getting MION FW versions
This commit moves us along further on the PCFS front, where SDIO,
and ATAPI are now probably properly being handled? At least for the
packet types that I know about. However, the big thing in this commit
is completely reworking the error structure for `cat-dev`.

This is due to the fact the error structures were being WAY TOOOO
MESSY. A couple problems with the existing strategy:

1. So many fields being turned into the same error type, means matching
   almost all APIs you would have to match way more error types than
   could ever be returned (matching errors becoming tedious!).
2. Most of the errors are stored 'far away' from the actual source code
   that generates those errors (small issue, but bugged me!).
3. 'Duplicate' Errors, e.g. there were many that were "invalid size",
   when we already had `FieldTooShort`/`FieldTooLong`. These errors were
   almost always worse than the FieldTooShort/FieldTooLong.
4. Many errors didn't implement `From`/`Into` for all the errors in the
   "upchain", so you usually had to manually wrap error types. Now
   almost all errors can be changed with just `Into`, `From`, and `?`.

All in all, error handling should be significantly cleaner all things
considered. There is a single case which is worse:

- Accepting a top level error type (e.g. `CatBridgeError`), but trying
  to parse out an inner error type (e.g. `mionps` & `mionparamspace`).
  Because error types are significantly more structured, this means more
  wrapping in that case, and thus uglier code.

Overall I'm not too worried about this, because most APIs do return the
smallest possible structured type they can. Most are not returning
`CatBridgeError`. Not to mention this specific error matching isn't used
very frequently at all (as evidenced by it only being in two specific
CLI's that have buggy behavior anyway). Given how much simpler the `?`,
`into`, and smaller error structures are.
- actually read all the fields from `FSEmul.conf` and respect them
  when needed.
- allow ctrl-c'ing out of serial wait task when connecting to
  `DEBUG_OUT`.
- add cli arguments for anything that can be specified in an env
  var/file to ensure per process overrides exist.
- scope out a bit more of the `boot` command when using a "default"
  setup (e.g. with SATA enabled, FFIO/CSR setup).
- add a load bearing sleep call between sending SDIO packets, so we
  don't overload the MION processing our devices (this is required to
  even be able to send `fw.img` because the MION is actually really
  terrible at it's job).
- allow legacy bridges to actually detect PCFS being enabled.
  - i still need to figure out ways to override all the args, probably
    more web setup calls but we'll see.
- fix latest clippy warnings
- upgrade to latest rust, and latest rust package resolver
- allow dlfs to recognize addresses that appear mid-way through a file.
- host filesystem now resembles more of a filesystem.
- implement a lot more of the PCFS Sata server, enough to complete a
  full MLC bootup and scan (we are now blocked on SDIO message types
  again).
- implement write_file for pcfs
- implement change_owner for pcfs
- fix up documentation
- work around case-insensitivity problems for linux with title ids.
  - windows is case insensitive, as is macosx, ext4 common linux however
    is case sensitive. most of the time copying between the two will
    just work. EXCEPT for title ids (which are expected to be
    lowercase).
@Mythra Mythra merged commit fa1ebdc into trunk Jan 14, 2025
15 checks passed
@Mythra Mythra deleted the dev/pcfs-frd-frd-frd-frd-frd-frd branch January 14, 2025 06:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant