Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up packet bundles and feature flags #294

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

TheDevMinerTV
Copy link
Member

No description provided.

@TheDevMinerTV TheDevMinerTV requested review from 0forks, Eirenliel and unlogisch04 and removed request for 0forks October 2, 2023 17:05
Copy link
Contributor

@unlogisch04 unlogisch04 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about the changes in featureflags. I can't review that. That template/class is something that i understand too less.

The following should be reviewed, i could not read the code from the other side so i dont know wht it expects:
sendPacketType
sendLong (uint64_t or int64_t)

src/network/connection.cpp Outdated Show resolved Hide resolved
src/network/connection.cpp Show resolved Hide resolved
src/network/connection.h Outdated Show resolved Hide resolved
src/network/connection.cpp Show resolved Hide resolved
src/network/connection.cpp Outdated Show resolved Hide resolved
src/network/featureflags.h Show resolved Hide resolved
@0forks
Copy link
Contributor

0forks commented Oct 2, 2023

The new packet bundling logic is not working as intended, sends empty bundles flooding the network when there's no data written between beginBundle() and endBundle(). Idle PPS is expected be around 5-10, I'm observing close to 1000.

image

I mentioned previously that we can't write the bundle header to the UDP library buffer from beginBundle() because an API to reset it doesn't exist. I suggest reverting to the original logic or making a dynamic buffer (or just large enough: 64 * MAX_IMU_COUNT) to hold the whole bundle, which is slightly less efficient. Either way it'll look messy I think.

bool Connection::beginBundle() {
memset(m_BundledPacket, 0, sizeof(m_BundledPacket));
m_BundlePacketInnerCount = 0;
MUST_TRANSFER_BOOL(m_ServerFeatures.has(EServerFeatureFlags::PROTOCOL_BUNDLE_SUPPORT
));
MUST_TRANSFER_BOOL(m_Connected);
MUST_TRANSFER_BOOL(!m_IsBundle);
MUST_TRANSFER_BOOL(beginPacket());
MUST_TRANSFER_BOOL(sendPacketType(PACKET_BUNDLE));
MUST_TRANSFER_BOOL(sendPacketNumber());

We could alternatively fix it by changing checks in the Sensor/SensorManager, but that would be less flexible, for example, if we want to wrap more than just sensor->sendData().

Maybe also worth moving the list of default firmware flags back into featureflags.h so it's more obvious that they can be marked as enabled when adding new features.

std::unordered_map<EFirmwareFeatureFlags, bool, EnumClassHash>
m_EnabledFirmwareFeatures
= {{EFirmwareFeatureFlags::B64_WIFI_SCANNING, true}};

@TheDevMinerTV TheDevMinerTV force-pushed the cleanup-feature-flags branch 2 times, most recently from c877176 to 69dcc7d Compare October 3, 2023 01:19
@TheDevMinerTV
Copy link
Member Author

I've moved the list of active features next to where they are declared.

Next time please use review comments ^^

@TheDevMinerTV TheDevMinerTV force-pushed the cleanup-feature-flags branch 2 times, most recently from 8fba600 to be6ba09 Compare October 8, 2023 20:18
@TheDevMinerTV
Copy link
Member Author

The new packet bundling logic is not working as intended, sends empty bundles flooding the network when there's no data written between beginBundle() and endBundle().

I fixed this now, it how has a buffer of the size of MAX_IMU_COUNT * 53 (53 being the size of the inspection packet).

@TheDevMinerTV
Copy link
Member Author

@0forks @unlogisch04 @Eirenliel please review

Copy link
Contributor

@unlogisch04 unlogisch04 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For me it seems to work. But i did not test it with trackers.
For ICM20948 there was a fix needed as they had a internal delay to not send too much updates.
For me the featureflags is something that could be improved more in the meaning more simple as 1 class / function that is seeded with different enums.

/*
The current incoming packet that is being handled
TODO: remove this from the class and make it a local variable
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This todo is open?
Forgot or does it not matter for this PR?

if (getBundleSize() + size > MAX_BUNDLE_SIZE) {
m_Logger.error("Bundled packet too large");

// TODO: Drop the currently forming packet
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean up and reset of the buffer?
I think we should check for length before we even add more bytes, so we could send a packetbundle, and open a new one for the one that did not fit.


// I hate C++11 - they fixed this in C++14, but our compilers are old as the iceage
struct EnumClassHash {
template <typename T>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not like the template here. As we do not save any programming bytes with it. Also the code understanding is more difficult in my meaning. I think that's a possible improvement for the future.

Copy link
Member

@ButterscotchV ButterscotchV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine, I am curious about the comments on the last review though and would appreciate an update

Comment on lines +176 to +180
/* `53` is the maximum size of any packet that could be bundled, which is the
* inspection packet. The additional `2` bytes are from the field which describes
* how long the next bundled packet is. If you're having bundle size issues, then
* you forgot to increase MAX_IMU_COUNT in `defines.h`. */
constexpr static size_t MAX_BUNDLE_SIZE = MAX_IMU_COUNT * (53 + 2);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change to MAX_IMU_COUNT * 64 (really 63 but it'll be aligned anyway), it'll drop packets in case of BNO+mag+tap with only 55 bytes. The comment also should reflect that it's the size of the whole bundle, not individual packets inside.

@unlogisch04
Copy link
Contributor

I did check the performance on that PR. It seems todo the relevant function (has) is about 8 times slower: It only gets called currently at each packet once. so about 100-120times a sec.

FeatureFlag1 = the new function
FeatureFlag2 = existing function
FeatureFlag3 = Raw && 2 uint8_t > 0

FeatureFlag1 Setuptime: 9 us
FeatureFlag1 100000     reads: 81500 us
FeatureFlag1 1      avg reads: 0.815 us
FeatureFlag2 Setuptime: 1 us
FeatureFlag2 100000     reads: 11258 us
FeatureFlag2 1      avg reads: 0.113 us
FeatureFlag3 Setuptime: 4 us
FeatureFlag3 100000     reads: 10049 us
FeatureFlag3 1      avg reads: 0.100 us

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants