Getting item information

[crept]

I had an idea of getting items and putting them in a format that eqemu can use. I am curious if this is even feasible. Would it be possible to, say, websuck all the items from Allakhazam's, parse the output, and arrange the information in a format that is useful for eqemu? I did some poking around in the source and as far as I could tell item information is stored in binary form in the DB. I didn't go much beyond that, though.

I know it's possible to get all the items from allakhazams quite easily, since their URL format is pretty simple. I've gotten a few items by hand crafting an URL for wget to download.

I believe if someone could post the format of the binary data in the item DB that would be enough for me to make the determination myself (or even just a header file with the info in it).

Note - if anyone were to write a script to do this, remember to put in a delay between each query so you don't tax their servers unnecessarily.

[Trumpcard]

I talked about this a long time ago, writing a java spider that could walk the sites and grab all item data. It wouldn't be that horribly difficult. The hardest part I would think would be getting it into the blob format to reinsert. Also, alot of characteristics arent going to be accounted for in the websuck, so you'd probably still have to tweak alot of them by hand changing default cases.

I've manipulated the blobs a bit through php, but that was just reading the data, not manipulating it. WC might have a better opinion. If we gave you a text file with the following template format,

Name:
Item Type: 1 hand slash
Weight: 6.0
Lore: Y
etc...

could someone come up with a parser that could recreate it into an sql statement in the binary blob format? If someone says yes to that, then writing a spider would be a piece of cake.. Just give it the base URL, have it search each page for those characteristics, then insert it into a .db file that could be run through a encoder to form an itemupdate.sql

Just my 2 cents.

[Lurker_005]

Well I ran across this a little while ago http://www.magelo.com/eq_item_info.html?num=15624 <-- the num is the proper item number in our DB also. At the very least it would allow us to get some info for the missing items that magelo has in it's DB Then when the real thing is collected with packetcollector the hand made item is just overwritten by the propper one. Perhaps some of the other info can be filled in from parsing stuff from other sources. This would include the Hackersquest items DB which seems to be in the propper binary format, but has 4 extra bytes after each item.

Note: I see this as useful to fill out missing items for the time being, not as a replacement for the packetcollected items. Also to get hand made items using the propper item number.

[crept]

Hmm, unfortunately the item stats in that magelo link are only in graphic form. Without some way to read the text in the graphic and convert it to text it won't be much use. We can, however, cross reference the "True Name" with whatever data we extract to get the ID number. The only contigency here is that both the names match up.

[Lurker_005]

From rightclicking on the item info and selecting view source:

<script type='text/javascript'>var spell_url='http://www.magelo.com/eq_spell_info.html';</script><script>var num_icon='0005';</script>Spell: Lesser Conjuration: Earth
<br>Effect: <script type="text/javascript">if(typeof(spell_url)!="undefined") document.write("<a href='"+spell_url+"?num=624' target='_parent'>Lesser Conjuration: Earth</a>  (Combat)"); else document.write("Lesser Conjuration: Earth");
</script><br>WT:  0.1    Size: SMALL<br>Class: MAG <br>Race: NONE<br>

so:
Item number was used to get the info,
icon='0005'
Spell: Lesser Conjuration: Earth
spell_num=624
Lesser Conjuration: Earth (Combat)
WT: 0.1
Size: SMALL
Class: MAG
Race: NONE

[crept]

We can use that URL to get spell info -

http://www.magelo.com/eq_spell_info.html?num=??

But I don't see a way to get item info in text form. Am I missing something?

[Lurker_005]

Right click on the item info graphic (brown leather area) and view source. It is a seperate frame.

And you don't really need the spell info, just the spell number.

Another example, Item 31233

<script type='text/javascript'>var spell_url='http://www.magelo.com/eq_spell_info.html';</script><script>var num_icon='0273';
</script>Akkirus' Mask of Warfare<br>MAGIC ITEM  LORE ITEM  NO DROP  <br>Slot: FACE  <br>AC: 30<br>
STR: +10  DEX: +10  STA: +10  CHA: +10  <br>WIS: +10  INT: +10  AGI: +10  
HP: +100  <br>MANA: +100  SV FIRE: +8  SV DISEASE: +8  SV COLD: +8  <br>SV MAGIC: +8  
SV POISON: +8<br>WT:  0.0    Size: TINY<br>Class: ALL<br>Race: ALL<br>