Some rekord box analysis

[teknopaul]

Hi i ran strings on some of the rekordbox applications and it is revealing of how they name their features. Some of these terms might help with writing docs and code.

Just an FYI:

They refer to the network as a link

InnerLinkAPI::getModelName I think is what you call Device Name, seems its the same for all devices onf the same model.

They refer to LinkMembers InnerLinkAPI::receiveLinkMember
InnerLinkAPI::requestPlayState

InnerLinkAPI::isPlayingATrackFromOurCollection
setBeatSyncMaster and beat "sync generally"

They seem to use the term ID instead of Device Number ref frameSendIdUseRequest, frameSendIdSetRequest,

those looks like names of some of the startup requests, they seem to be refereed to as Discovery in Pioneer executables.

with regard to protocol analysis, I noticed three types of communication.

PSvLinkTCPClient Thread
PSvLinkIPC Thread
PSvLinkUDP Thread

on these ports, some of which have not been reversed.

PSvLinkLog::portchk
%s OK 2049
PSvLinkLog::portchk
%s OK 50000
PSvLinkLog::portchk
%s OK 50001
PSvLinkLog::portchk
%s OK 50002
PSvLinkLog::portchk
%s OK 50004
PSvLinkLog::portchk
%s OK 50111

2049 is TCP NFS, so potentially some of these devices use NFS protocol , they seem to run a modified linux / busybox based on the opensource code they are publishing on their website.

I nned to get a network HUB to be able to snoop more, naturally, rekordbox will have features that player s don't.

[brunchboy]

Hi, thanks for the note! Yes, the players serve their USBs to each other over NFS, that ends up being the only way to get track metadata when there are four players (older than the CDJ-2000) on the network. My implementation of that is Crate Digger.

I wonder what they mean by the “IPC” thread. The UDP thread is clearly for the UDP packets that send status and discovery information, and the TCP thread is likely for the dbserver connections which provide access to the DeviceSQL database.

I think “device number” is a much more meaningful way to describe that protocol value than “ID”, although sometimes “channel number” is even better, as when connected to a DJM, you want the player to use the same mixer channel number as its device number for ease of understanding the relationship between what is playing and the level/filter/effect knobs you are adjusting. But, given how painfully a lot of the protocol is designed, I am not surprised to hear their code uses some less-than-ideal names for things. 😄

If you figure out any details of more of the ports and packets that can be usefully applied to running shows in Beat Link Trigger, I’d love to hear about it. In any case, it is always a pleasure to learn about people investigating this space.

[brunchboy]

Oh, and I found the NETGEAR 5-Port Gigabit Ethernet Smart Managed Plus Switch (GS105Ev2) to be a very handy and reasonably-priced tool for watching the interaction between devices, in case it is available in your region as well: https://www.amazon.com/gp/product/B00HGLVZLY/

[teknopaul]

OK sounds like you are ahead of me re: nfs. Do you know anything about what goes on on 50004 or 50111 ?
I had a run through the strings and it has some SQL commands visible, it seems a lot is already known about that.
My goal is to get syncing and control of bpm to two decimal places .00 on XDJs.
I'm writing a lib in C to integrate with LMMS and ALSA, and hopefully to run on a raspberryi pi. It would be fun to sync CDJ-350s with nexus tech

[brunchboy]

Everything I know about is written up in the protocol analysis document. I haven’t done anything with the port 50004 stuff, my DJM-2000 nexus does not support it, but what we know is described here: https://djl-analysis.deepsymmetry.org/djl-analysis/packets.html#_port_50004_packets

I have not seen or heard anything about port 50111. Let us know if you can figure anything out!

I implemented bpm and beat sync, bidirectionally to players, in my Beat Link library.

[teknopaul]

Hi I have some more info about the boot sequence (I'm interested in how ID allocation works for devices without keyboards)

I believe the message types are in pairs, even numbers are a request odd numbers are a response to that message type.

The message you call "Second-stage" (type 0x02) I believe what Pioneer call an IdUseRequest message. It is broadcast on 50000 to 192.168.1.255.

If a CDJ already has this ID number assigned there is an IdUseReply message (type 0x03) unicast on port 50000 back to the player that tried to take a preassigned ID. Took a while to spot this because I was not listening to unicast.

This is a dump of a reply from a player 04, (len 0x27) not sure what the 01 01 could mean. (edit: first 01 is N the packet index identifier from the offending request)

51 73 70 74 31 57 6d 4a 4f 4c 03 00 58 44 4a 2d
31 30 30 30 00 00 00 00 00 00 00 00 00 00 00 00
01 02 00 27 04 01 01

The 01 02 00 seems standard across all messages in this part of the protocol.

In the boot sequence, if an XDJ receives this message, it tries a different player ID immediately in its next broadcast IdUse request message.

Seems this is how automatic player ID assignment works. Any CDJ can claim an ID with a IdUseReply and the booting CDJ will respect that.

The next set of 3 messages I believe is what Pioneer code calls IdSetRequest, i.e. it tells the cluster this player has agreed on an ID.

If you don't respect it, (or miss the reply) and boot anyway, on the wrong ID and send a type 0x04 with an ID that someone already owns it broadcasts to 50000 a ID clashing message (type 0x08, so far undocumented) to tell the cluster we have a problem. Potentially this is called ConfigNotify or CompatiRequest. The player id lights start flashing on all the players. All player enter the ID assignment mode again starting by sending out IdUseRequest messages again on broadcast 50000 asserting their IDs.

I guess a polite player should listen on 50000 unicast and respect existing devices, it if gets an ID clash it should presume it missed one and re init along with the rest of the cluster.

The ID clash error message that is broadcast looks like this. (some bytes converted to ASCII already, c0 a8 01 3c is its own IP)

Q_ s_ p_ t_ 1_ W_ m_ J_ O_ L_ 08 00 X_ D_ J_ -_
1_ 0_ 0_ 0_ __ __ __ __ __ __ __ __ __ __ __ __
01 02 00 29 04 c0 a8 01 3c

it asserts that I have device number 04 on this IP and someone is trying to take that over and they should be ignored. I think this is the IdUseResponseFromBlk message we can call it the "get oorf my laaand" message :)

I've got a router coming in the mail to analise two cdjs talking to each other, but for the moment I can reliably force this issue so I should be able to work on a solution to the AUTO Id Assignment issue already.

Is there anyway I can submit content patches to your protocol pdf I can format the above to something more formal when I have a fix working.

[teknopaul]

This is probably why there are 3 messages, that enables up to 4 players on the network to auto assign

[brunchboy]

Thanks for the additional discoveries, this is great information!

Where are you finding the protocol PDF? It hasn’t been delivered as a PDF for nearly a year now, so you are looking at outdated information. It is now an Antora site hosted at https://djl-analysis.deepsymmetry.org/ (and the old PDF download URL redirects to this site, I just verified, did you find it somewhere else)?

The source for the site is in this repository, so you can submit pull requests in the normal GitHub manner. That’s probably the best way to do it, because once you have the site building properly locally, you can check how your changes look in context before submitting a unified pull request. However, like most Antora sites, each page has an “Edit this page” link at the top right which lets you use GitHub’s web-based editor to adjust the source and submit a pull request directly. If you’re confident enough in your AsciiDoc and Antora skills to do that blind, it’s a quick option. But I’d suggest trying it the more traditional way with a clone of the repository you can preview locally a few times first, just to be sure.

If you’re not already familiar with writing rich documentation in AsciiDoc, the Asciidoctor documentation is a good starting point. And the tool I created for building the packet diagrams is called bytefield-svg, and of course I wrote up a documentation site for that as well.

[teknopaul]

errr yep, I guess I have had this project on my todo list for about a year :$

[teknopaul]

FYI IdUseReply packet 0x25 matches N the counter of the IdUse outgoing message, so its possible to correlate a response to a specific request.

I'm pretty sure the 300ms delay and the fact that there are 3 messafes is to facilitate a 4 player netowrk autoconf. the 300ms is to wait for unicast reply, I've not seen replys to the first two messages and I cant see what errors could occur but perhaps old devices used to auto-assign IPs instead of using APIPA and dhcp?

[teknopaul]

meh, https://djl-analysis.deepsymmetry.org/djl-analysis/packets.html#_port_50000_packets looks like kind/type are not unique

[brunchboy]

Say, @teknopaul—since you are looking at the actual binary code, maybe you can help with an annoying problem we have just been discussing on the GItter channel. It looks like in rekordbox 6 they finally started including song structure (phrase information) tags in the exported analysis .EXT files, presumably so that the CDJ-3000 can talk to their lighting interface without rekordbox in the loop. This would be fantastic for Beat Link Trigger to be able to tap into, as it could provide automated lighting looks even for tracks that have not been manually configured. But it looks like they have garbled or encrypted the data somehow, because it has the same shape, but important values suddenly are very wrong. Also, this data has been removed from the rekordbox library in the filesystem, and moved into the encrypted database. So this seems clearly an effort on their part to prevent other people from taking advantage of the data. Big sigh.

Is there any hope that you might be able to find the code which exports PSSI tags to removable media, and figure out how it is transforming them? Or where the CDJ-3000 firmware is reading them, and how it undoes that?

[brunchboy]

@teknopaul we have figured out the “encryption” algorithm and key generation rules they are using, you can see the details in the discussion starting here: Deep-Symmetry/crate-digger#3 (comment)