Proposed Network Meta-Language

daeken_bb · #1 06-05-2005, 08:31 PM

This is a proposition for a meta-language for networking. This would provide us with a way to build packet parsers with little to no hassle. These can be compiled to C code (which would be a struct, or series of structs, and a function that builds the packet or reads the data into a usable struct) or can be loaded/modified on the fly for development.

Syntax:

Code:

struct struct_name (optional size) {
	type	item_name;
	type	item_name = default_value;
	type	item_name = 's'; // This works as in C.
	type	item_name[count];
	type	item_name[count] = {5, 6, 7, 8};
	type	item_name[count] = 'non-terminated string';
	type	item_name[count] = "null-terminated string";
	struct struct_name	item_name;
	struct struct_name	item_name[count];
};

You can also use a map structure to rename items without actually changing their name, this is useful for keeping structures named as they are elsewhere, while keeping compatibility with existing code.

You can use struct items as the count for an array, unlike in C. However, this means it can not be translated to a single array, and must be processed separately. This should not be any more of a performance hit than doing it by hand.

Maps are defined as such:

Code:

struct ... {
...
} map {
	Access_Name: name_in_struct;
};

Examples:

A simple struct and map.

Code:

struct NameApproval (76) {
	char	name[64];	/* 00000000 */
	uint32	race;		/* 00000064 */
	uint32	class;		/* 00000068 */
	uint32	deity;		/* 00000072 */
} map {
	Name:	name;
	Race:	race;
	Class_:	class;
	Deity:	deity;
};

A dynamic length string.

Code:

struct DynamicLength {
	uint32	length;
	char	name[length];
};

Any input would be welcome.

Thanks,
Lord Daeken M. BlackBlade
(Cody Brocious)

**fathernitwit** · #2 06-07-2005, 03:09 AM

im not gunna get too involved in this discussion, but I will throw in what I took away from me and Doodman talking about this a while back.

Basically the best way to achieve independance from struct changes in the client is to do what I did for opcodes: Make all the game code use an internal representation of the struct which will not change, and therefore not need recompile. Then have some agent to convert from this internal representation to the actual net-layer representation when you send or receive the packet.

Your meta layer would address some of this in that it could generate the internal structures, and assuming it supported different named maps for each 'patch' you supported then it could generate most or all of the translation routines as well.

The major problem with this scheme is that you have to keep in mind what a struct change normally means: they added something to the packet. The internal packet representation would work fine if they scrambled or shortened a packet, but that is rarely the case. In the case of addition, in order to do something useful with the additional field, you would have to also add something to the internal representation of the packet to get that additional information down to the packet translator, which means recompile. Granted, if you just wanted to leave the new space as 'unknown' then you could get away without recompiling. But this kinda limits the utility of it to pure testing and ignoring the new fields.

Also keep in mind that there are many dynamic length packets which do not contain their length at all, but instead rely on (possibly many different) null terminated strings to dictate their length. Also arrays of dynamic length structs. Both of these are getting more and more common in eq1, so I can only imagine they are coming from eq2-isms.

I am sure there are other ways to pull this off as well, such as using some dynamic structure in memory, and a dynamic converter, but your talking about significant overhead in those situations.