New DSL for MachOStructure

[woodruffw]

This is just for collecting my thoughts and getting feedback on a DSL for MachOStructure (and, by inheritance, just about every Mach-O data-oriented class in ruby-macho.

The current "DSL" for a Mach-O structure looks something like this:

class SomeImportantStruct < MachOStructure
  attr_reader :something
  attr_reader :something_else

  FORMAT = "L=2" # something and something_else are "L", meaning int32s
  SIZEOF = 8 # 2 x sizeof(int32)

  def initialize(something, something_else)
    # ...
  end 
end

This is incredibly repetitive, bug-prone, and difficult to read (there's no immediate way to determine the size of each field).

Something like this would be substantially better:

class SomeImportantStruct < MachOStructure
  field :int32, :something
  field :int32, :something_else

  # FORMAT, SIZEOF, and instantiation are automatically generated from the field definitions
end

Extremely early versions of ruby-macho actually had something like this, using CStruct from this repo, but I dropped it for reasons that I don't remember (probably having to do with me being unfamiliar with ruby DSL writing at the time).

This will probably require a large amount of refactoring and breakage, so I don't think it'll make it into 1.1 (or even 1.x, it might be destined for a major bump).

[aidansteele]

Are you familiar with the bindata gem? It sounds like exactly what you're after. I've used it to great success with projects very similar to this gem.

[woodruffw]

@aidansteele That looks like exactly what I want, thank you!

[aidansteele]

No worries 👍 While I'm thinking of it, there's also this repo I put up a while ago:

https://github.com/aidansteele/osx-abi-macho-file-format-reference

It's just a markdown version (+ original Apple PDF) of the OS X ABI Mach-O File Format Reference. This document used to be on Apple's website until relatively recently, but seems to have dropped off the face of the Earth. Which is a shame, because it's probably the most useful reference document I can think of for this project.

[woodruffw]

This document used to be on Apple's website until relatively recently, but seems to have dropped off the face of the Earth.

Yep, I know exactly what you're talking about. That reference + the cctools source basically made ruby-macho possible. Thank you so much for archiving it!

[luikore]

I think a DSL like construct is super concise and expressive, much better than bindata

You can take a look at these 2 files to see how simple it is to parse and generate macho and code signature in python:

https://github.com/comex/cs/blob/master/macho.py
https://github.com/comex/cs/blob/master/macho_cs.py

[BrewTestBot]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[apainintheneck]

I'd be interested in working on this. I assume the main thing is just to generate the sizeof, format and initialize() portions of the class. I'm pretty new to Ruby dsl design but I'll see what I can code up.

[apainintheneck]

I was thinking about something like the following which would replace the attr_reader, format, sizeof, and initialize parts of the current class definition.

class SomeImportantStruct < MachOStructure
  field :something, :uint32
  # Strings also need custom size
  field :something_else, :string, :size => len
  # Occasionally a custom bitmask is used
  field :another_one, :int32, :mask => bitmask
  # Finally custom types are sometimes necessary
  field :one_more, :custom, :size => len, :fmt => format_string
end

[woodruffw]

Something like that looks reasonable to me. For `:string`, we can probably do something like `:string => :lcstr` to indicate that the string should be parsed as a `struct lc_str`, since that's the most common (only?) case.

…

On Sat, Mar 26, 2022 at 11:05:06AM -0700, Kevin wrote: I was thinking about something like the following which would replace the attr_reader, format, sizeof, and initialize parts of the current class definition. class SomeImportantStruct < MachOStructure field :something, :uint32 # Strings also need custom size field :something_else, :string, :size => len # Occasionally a custom bitmask is used field :another_one, :int32, :mask => bitmask # Finally custom types are sometimes necessary field :one_more, :custom, :size => len, :fmt => format_string end — Reply to this email directly, [1]view it on GitHub, or [2]unsubscribe. You are receiving this because you were assigned. Message ID: ***@***.***> References 1. #70 (comment) 2. https://github.com/notifications/unsubscribe-auth/AAXK4CQVCEA6L2KXRPHWEMDVB5GVFANCNFSM4C5SUEBQ

[apainintheneck]

Oh yeah, I hadn't gotten around to thinking about the fields that don't actually add anything to the format or sizeof attributes. The :string I was talking about above was the null terminated string that parses with the letter Z that's used in the Section classes.

After a quick look around it seems like the main ones I was missing were the lc_str and view attributes that are added to a few classes. I'm almost inclined to just make those their own types.

class SomeImportantStruct < MachOStructure
  field :name1, :view
  field :name2, :lcstr
end