Changes in FSD?

[blitzmann]

cfg.icons

When trying to fetch icon data (which is specified by reverence as being in FSD format):

from reverence import blue

eve_path = r"C:\EVE\SharedCache\tq"
res_path = r"C:\EVE\SharedCache"

rvr = blue.EVE(eve_path, sharedcachepath=res_path, server="tranquility")
cfg = rvr.getconfigmgr()

for key, value in cfg.icons.iteritems():
    print key, value

Results in the following for the vast majority of the data:

0 FSD_Object(iconFile:'',obsolete:NULL,iconType:NULL)
15 FSD_Object(iconFile:'',obsolete:NULL,iconType:NULL)
16 FSD_Object(iconFile:'',obsolete:NULL,iconType:NULL)
21 FSD_Object(iconFile:'',obsolete:NULL,iconType:NULL)
22 FSD_Object(iconFile:'',obsolete:NULL,iconType:NULL)
26 FSD_Object(iconFile:'',obsolete:NULL,iconType:NULL)
27 FSD_Object(iconFile:'',obsolete:NULL,iconType:NULL)
28 FSD_Object(iconFile:'',obsolete:NULL,iconType:NULL)
29 FSD_Object(iconFile:'',obsolete:NULL,iconType:NULL)
30 FSD_Object(iconFile:'',obsolete:NULL,iconType:NULL)
...

Note that iconFile is an empty string. There are entries that are used for clothing assets it seemed, but the values seem to have encoding issues?

...
10666 FSD_Object(iconFile:'c\x00\x00\x00res:/UI/Asset/mannequin/topmiddle/4212_female_TopMiddle_ShirtGF01_Types_ShirtGF01_p_darkcaldari.png',obsolete:NULL,iconType:'c\x00\x00\x00res:/UI/Asset/mannequin/topmiddle/4212_female_TopMiddle_ShirtGF01_Types_ShirtGF01_p_darkcaldari.png')
10667 FSD_Object(iconFile:'b\x00\x00\x00res:/UI/Asset/mannequin/topmiddle/4213_female_TopMiddle_ShirtGF01_Types_ShirtGF01_p_desertcamo.png',obsolete:NULL,iconType:'b\x00\x00\x00res:/UI/Asset/mannequin/topmiddle/4213_female_TopMiddle_ShirtGF01_Types_ShirtGF01_p_desertcamo.png')
10668 FSD_Object(iconFile:'Y\x00\x00\x00res:/UI/Asset/mannequin/topmiddle/4214_female_TopMiddle_ShirtGF01_Types_ShirtGF01_red.png',obsolete:NULL,iconType:'Y\x00\x00\x00res:/UI/Asset/mannequin/topmiddle/4214_female_TopMiddle_ShirtGF01_Types_ShirtGF01_red.png')
10669 FSD_Object(iconFile:'[\x00\x00\x00res:/UI/Asset/mannequin/topmiddle/4215_female_TopMiddle_ShirtGF01_Types_ShirtGF01_white.png',obsolete:NULL,iconType:'[\x00\x00\x00res:/UI/Asset/mannequin/topmiddle/4215_female_TopMiddle_ShirtGF01_Types_ShirtGF01_white.png')
...

This seemed to work fine a few days ago. Not sure if the FSD format changed or if the data was moved somewhere else

cfg.graphics

I don't personally use this resource, but I figured I would try to test it. This returns one complete value as far as I can tell, then exception happens:

10 FSD_Object(graphicFile:'*\x00\x00\x00res:/dx9/model/worldobject/planet/moon.red',sofRaceName:NULL,sofHullName:NULL,sofFactionName:NULL,collisionFile:NULL,isisIconPath:NULL,animationStates:NULL,ammoColor:NULL,albedoColor:FSD_NamedVector(r:6.44597293589e-44,g:5.88545355016e-44,b:0.000928483088501,a:0.000236884443439),emissiveColor:NULL)
38 

Traceback (most recent call last):
  File "C:/Users/Ryan/Copy/Git/blitzmann/Phobos/test.py", line 11, in <module>
    print key, value
  File "build\bdist.win-amd64\egg\reverence\fsd.py", line 760, in __str__
ValueError: _int32_from requires a buffer of at least 4 bytes

[blitzmann]

I had a bit of time this morning waiting for a hurricane to pass over my area, found out a bit more how reverence works. A lot of it is still over my head, however I have a better understanding of it.

There are still a few things that produce errors when it comes to int/uint functions. I've looked at the C code but don't immediately understand what may cause it. I'll open another issue on that later. But, for the icons / graphics themselves, which this issue is about, I noticed this:

editorFile: "eve/staticData/iconIDs/iconIDs.staticdata"
editorFileSchema: indexedFile
runtimeSchema: indexedFile

schemas:
  element: &elementType
    type: object
    attributes:
      iconFile:
        type: resPath
        isOptional: true
        export: true
        extensions: ['png', 'jpg']
      description:
        type: string
        isOptional: true
        usage: Editor
        export: true
      obsolete:
        type: bool
        isOptional: true        
      iconType:
        type: string
        isOptional: true
        usage: Client


  # This is the indexed data file that the client uses
  indexedFile:
    type: dict
    buildIndex: true
    keyTypes:
      type: int
      IDGenerator:
        type: UIDFromWebserver
        server: "https://fsdid.tech.ccp.is/GetCounter/IconID"
        key: value
    valueTypes:
      *elementType

This is the iconIDs.schema res file. Note how there is an IDGenerator pointing to a webserver - I am assuming that the client points to this to get the icon information that is missing, and reverence doesn't handle this at all from what I can see. Same with the graphics . I've tried to set up packet capture to capture traffic on this host (10.91.255.3), with no luck so far.

[ntt]

Heh, that's new.

Yeah, the FSD stuff has been broken for a while. It's probably the most complicated part of this whole thing; it's a rather obscure format.
Even when it was still implemented in Python in the game, it was pretty hard to figure out how it worked.

Sadly, I currently lack the motivation to invest the large amount of time I'd need to figure out the format again to fix the problems with it (CCP seems to keep changing it, grrr)

Also, even if the FSD decoder was fixed, I would not likely make reverence access any game servers. That particular server is clearly not an API for 3rd party developers to use. (Maybe if CCP said it was ok, I'd consider it)

[blitzmann]

Yeah, I don't expect for it to get fixed. I am mostly playign around with it and looking at the various formats CCP uses. I am interested to know where this data comes from / is saved though. I feel like it has to be cached somewhere - they wouldn't make an api call every time they need icon info.