Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Response to concerns about NanoKVM security #301

Open
Zepan opened this issue Feb 6, 2025 · 34 comments
Open

Response to concerns about NanoKVM security #301

Zepan opened this issue Feb 6, 2025 · 34 comments

Comments

@Zepan
Copy link
Contributor

Zepan commented Feb 6, 2025

In apalrd's recent video (https://www.youtube.com/watch?v=plJGZQ35Q6I), apalrd raised several security concerns regarding NanoKVM.
We have carefully watched the video—some parts are fair, while others result from an incomplete understanding of the product.
To help users better understand these concerns, we are providing a unified response under this issue. We welcome objective discussions.

Security vs. Usability Trade-off

Before discussing the details, we want to clarify a fundamental fact: security and usability are inherently contradictory to some extent. If one pursues absolute security, usability will be significantly compromised. apalrd represents a group of users who prioritize security to the extreme. However, based on our customer support records, a large proportion of users care more about ease of use.

For example, we receive multiple emails daily from users who have forgotten their self-set passwords and need help recovering them. apalrd also pointed out that our page "reminds" users to change their passwords rather than "forcing" them to do so, considering this a security risk.

In fact, we did consider forcing password changes during the initial product design. However, after shipping the alpha version, we found that a significant proportion (perhaps over 50%) of our users are beginners. If we force them to change their passwords, they are likely to forget them. Additionally, those who forget their passwords typically do not read the wiki (which contains instructions for physical factory reset). This would leave them unable to use the product or require them to send support emails.

Thus, we chose to remind rather than force users to change their passwords. apalrd mentioned that many IoT devices have similar security issues, but aside from actual security risks, some design choices are compromises for usability.

Responses to Other Concerns Before Discussing Security Details

We understand apalrd’s strong focus on security. However, as a product designed for a wide range of users, we must balance security and usability. We believe different user groups have different needs, so we made trade-offs in our product design accordingly. If apalrd had carefully read the wiki before using the product, some misunderstandings in the video could have been avoided.

Responses to Non-Security Design Issues

  1. Does the full version require disassembly for reflashing?
  • No. We considered factory reset and reflashing methods in the design, as detailed in the wiki. By holding the BOOT button while powering on via USB, the device enters flashing mode. At this point, U-Boot enters UMS mode, allowing users to flash the image via dd or BalenaEtcher.
  1. Is using Type-C for ATX power control an abuse of USB cables?
  • Similar products use RJ11 for ATX power control, but this has two issues:
    1. Size: While RJ11 is acceptable for PiKVM, it is too large for NanoKVM.
    2. Availability: RJ11 cables are not always readily available at home. If users lose them, they must pay extra for replacements.
  • We specifically chose the smaller Type-C interface, allowing users to use readily available A-to-C cables of flexible lengths.
  • Regarding apalrd’s concern about potential damage from plugging in a normal USB cable, we considered this issue during design. Our implementation ensures that even if a standard USB signal is inserted, it will not cause electrical damage.
  1. Issues with PCIe naming and installation
  • The name “PCIe” was chosen based on a Twitter poll. Although it does not use PCIe communication, users still preferred this name for clarity.
  • apalrd did not carefully read the wiki or fully understand the PCIe version’s design. He assumed it was only mechanically fixed and did not address wiring or electrical functions.
  • The PCIe version was specifically designed for ATX case installation. While it has an external USB-C port, our recommended installation method is internal, using the provided DuPont cables to connect to the internal USB header, avoiding extra external wiring.
  • The PCIe slot is not just for mechanical fixation—NanoKVM draws power from PCIe 3.3Vaux, which remains powered even when the PC is off. Thanks to NanoKVM’s ultra-low power consumption (<1W), it can run continuously from 3.3Vaux without external power.
  • In fact, NanoKVM-PCIe requires only an HDMI connection to function externally, contrary to apalrd’s claim that it still needs multiple cables.
  • In an earlier design, we even considered connecting the PCIe wake# signal for wake-on-LAN functionality but later removed it based on overall usage considerations.
  1. Misunderstanding between system images and app versions
  • In the video, apalrd mentioned that the system image was from three months ago and outdated. However, he confused the system image with the app version.
  • The system image serves as a stable base. Once the fundamental drivers are in place, it does not need frequent updates.
  • However, we continuously update the app, incorporating user feedback, feature enhancements, and bug fixes—even modifying original system files when necessary.
  1. Misinterpretation of the open-source repository’s purpose
  • apalrd assumed we rely on the community for development. In reality, most PRs are for UI language translations, with very few for core functionality (for which we offer free new products as rewards).
  • We never intended for the community to take over development. Based on our experience with many prior open-source projects, commercial product development cannot rely on uncertain community contributions.
  • The main reason for open-sourcing NanoKVM is transparency—to prove there are no "Chinese backdoors." As apalrd’s own analysis confirmed, there was no evidence of intentional backdoors.

Response to Security Concerns

To address the security-related concerns, I have categorized all the issues and will respond accordingly.

Design Trade-offs for Usability and Availability

  1. Password Change Policy
    -As mentioned earlier, we chose to "remind" users to change their passwords rather than "force" them to do so.
  2. Forced DNS Modification
  • Some users in certain regions experience domain resolution failures with their local DNS providers, preventing access to Sipeed’s servers and causing update failures. To ensure availability, we enforced the use of reliable DNS services. While this decision may raise security concerns, it was made purely to guarantee usability for all users.
  • The issue you mentioned—"installation failure due to network restrictions in China"—is an example of such network-related problems.
  • Once again, security and usability are often in conflict, and we aim to strike a balance. Some harmless actions may cause unnecessary concerns.
  1. Bundled Outdated Version of Tailscale
  • During the Alpha hardware release, we did not preinstall Tailscale in the image. However, some users in certain regions were unable to access Tailscale's official website, so we used a CDN to distribute a fallback version.
  • In the final official image and hardware versions, this fallback link is not needed.
  • This was purely a usability-driven decision—just because your network can download Tailscale without issues, it doesn't mean all users around the world can do the same.

Non-Security Issues / Other Feature Enhancements

  1. Default Installation of tcpdump and aircrack
  • These packages were enabled by default in the original SDK for the SG2002 platform, and we initially retained the default configuration without much attention.
  • However, these tools have proven useful for diagnosing network issues users encounter. This has been evident in comments under your video and discussions on GitHub issues.
  • Additionally, our PCIe version includes WiFi support, and to avoid fragmenting the image versions, we provide a unified firmware image. As a result, some software packages may be present even if they are not immediately needed for a specific hardware variant.
  1. NanoKVM Lacks MFA Support
  • This is something better suited as a "TODO" item, and we will update the README roadmap accordingly.
  1. Closed-Source Libraries
  • This issue has already been addressed in the corresponding GitHub issue.
  • NanoKVM is a side project of MaixCAM, and for rapid development, we reused some proprietary MaixCAM libraries.
  • However, these libraries only handle chip-specific video encoding/decoding and do not contain anything concerning.
  • In fact, community developers have already reverse-engineered and reimplemented the necessary functionality.
  • In the next version, we will remove these dependencies to eliminate unnecessary speculation.

Security Concerns Stemming from Tailscale or Network Configuration

  1. NanoKVM as a Router
  • The Tailscale startup script sets ip forwarding = 1, effectively turning the device into a router.
  • Since enabling Tailscale automatically configures this setting, we provide users with the option to enable or disable Tailscale.
  • Users can also manually remove /etc/sysctl.d/99-tailscale.conf if needed.
  1. NanoKVM Identified as an "InternetGatewayDevice"
  • It is enabled by default in the original SDK for the SG2002, we can disable it next time

Beneficial Security Recommendations

  1. Potential Issues with SSH Enabled by Default
    Initially, NanoKVM was designed as a product for homelab developers. Therefore, we set the product to "Developer Mode" by default, enabling SSH to facilitate network diagnostics and recovery for developers. Many of our previous designs and configurations, as well as our earlier products like SBCs and AI Cameras, were also tailored for developer users. However, as NanoKVM reaches more general users, it is indeed time to default to a "Regular User" mode, with "Developer Mode" requiring manual activation.

  2. Hardcoded Keys in TypeScript
    Yes, this was an improper practice, a flaw left over from the early rapid development stage. We plan to move the keys into configuration files in the next version, which is expected to be released this month. This improvement will further enhance product security. However, it is worth noting that the primary security risk associated with this issue is man-in-the-middle attacks, meaning that the risk arises mainly when your network is untrusted or malicious. In such cases, the number of potentially compromised devices around you may be greater than expected.

  3. Lack of APP Update Verification

Similar to the previous issue, we will add verification in the next version. The primary risk also comes from man-in-the-middle attacks.

Summary

We are always committed to finding the best balance between security and usability and appreciate the feedback from apalrd and other users.
We will continue improving NanoKVM to ensure it meets the security needs of advanced users while remaining simple and user-friendly for beginners.
We welcome all users to transparently discuss and provide suggestions in an open-source environment. Whether the concerns are genuine security issues or security concerns influenced by bias, they can be properly addressed in a transparent, open discussion.

Additionally, we recommend users read our Wiki documentation before using NanoKVM to fully understand its features and usage.

For any questions, our technical support team is always available: [email protected].

@Zepan Zepan changed the title Response to apalrd's concerns about NanoKVM security Response to concerns about NanoKVM security Feb 6, 2025
@alexhk
Copy link

alexhk commented Feb 6, 2025

When you say "next version", are you talking about a new hardware (NanoKVM Pro), or a software update for current NanoKVM users?

@mcsrobert
Copy link

This feels really dismissive. Let's take the hardcoded DNS servers as an example, but this applies to multiple issues responded to here.

Hardcoded DNS is a pretty obvious bad practice. Your reply is: "Some users in certain regions experience domain resolution failures with their local DNS providers."

Then why not have a sane default and offer those few users who need to take extra steps the ability to easily change the DNS server?

Just because users in China are behind a firewall, doesn't mean that by default this device should ignore the DNS server in my network.

@wj-xiao
Copy link
Collaborator

wj-xiao commented Feb 6, 2025

When you say "next version", are you talking about a new hardware (NanoKVM Pro), or a software update for current NanoKVM users?

The software update.

@wj-xiao
Copy link
Collaborator

wj-xiao commented Feb 6, 2025

This feels really dismissive. Let's take the hardcoded DNS servers as an example, but this applies to multiple issues responded to here.

Hardcoded DNS is a pretty obvious bad practice. Your reply is: "Some users in certain regions experience domain resolution failures with their local DNS providers."

Then why not have a sane default and offer those few users who need to take extra steps the ability to easily change the DNS server?

Just because users in China are behind a firewall, doesn't mean that by default this device should ignore the DNS server in my network.

We will remove the modification operation and no longer modify the DNS configuration file.

@sean-wilkinson
Copy link

Hardcoded Keys in TypeScript
Yes, this was an improper practice, a flaw left over from the early rapid development stage. We plan to move the keys into configuration files in the next version, which is expected to be released this month. This improvement will further enhance product security. However, it is worth noting that the primary security risk associated with this issue is man-in-the-middle attacks, meaning that the risk arises mainly when your network is untrusted or malicious. In such cases, the number of potentially compromised devices around you may be greater than expected.

Why move the keys to a config file? The problem here is the fact that the password is encrypted and can be decrypted with a key.

  1. The password and salt should be hashed by the browser using a one way hash function such as sha256.
  2. The hash is sent over the network
  3. The Server (NanoKVM) should compare the hash with the hash stored locally.
  4. If the hashes match authentication was successful.

Employing Symmetric-key cryptography such as AES256 for login functionality such as this is very suspicious as it allows a man in the middle to decrypt the password if they have the key.

Using a one way hash function does not allow a threat actor to recover the password unless they have a very VERY large GPU farm to crack the hash.

These are basic security fundamentals.

@PatrickHuetter
Copy link

Dear Sipeed Team,

I assume that due to the recently released video, you will now receive increased feedback as well as critical messages—some of which are justified. However, I am convinced that you can use this situation as a great opportunity to become even more successful in the long term.

My advice: Support the open source community by releasing all the information needed to build a clean and efficient open source firmware. You will see that many more people will be willing to purchase your hardware because that is your unique selling point!

Personally, I would have bought the nanoKVMs much earlier if the open source aspect had been clearly resolved and everything had been made open source. Don’t be afraid to build something great together with the community—and don’t fear competition. If you deliver top-notch hardware while quickly and sustainably fixing the software issues, your project will be celebrated and widely adopted.

I truly appreciate the price/performance ratio of your hardware—even compared to the JetKVM. Now it’s up to you to optimize the software and release it fully open source. Be smart and let the community help you.

Best regards!

@ddscentral
Copy link

I personally find apalrd’s security concerns to be perfectly valid, given what type of device this is. This is a KVM device which is used to REMOTELY CONTROL a computer.
In my opinion, security and transparency should be your top priorities, even if they can sometimes come at a cost of ease of use. This device is not a toy to play with. Well, unless you want to treat is as such.

I have acquired several NanoKVM devices but so far I only put one of them in service for testing/tinkering purposes.
I have serious doubts about putting others into service with the software in it's current state. Even though I do like the hardware.

@gshpychka
Copy link

Why don't you respond to the specific issues in this repo, some of which are about the concerns you're addressing?
Opening a tracking issue is okay, but you should respond to the issues as they're opened, not wait for a youtube video to address them in a new and separate issue.

#245
#250
#289
#192
#248
#295
#197
#294

@apalrd
Copy link

apalrd commented Feb 6, 2025

I didn't pull my comments out of thin air. I read all of the Github issues that have been accumulating here for months before making the video, and covered the ones I thought were the most significant.

The lack of action on these issues (here, on Github) speaks volumes to your commitments to security in the product.

@TheSp1der
Copy link

Honestly, I don't think the review by apalrd's video was fair. I knew going in that this device was from a possible suspect source, in reality or not, but let's be clear everything on my network is a suspect. So, I treat it that way. Using sensationalized words and phrases like "Backdoor" and "Victim's computer" seems a bit unfair when there was no evidence to indicate any of that was true.

I was suspect of these and isolated them, but, honestly, Google devices make more scary calls than this thing does.

I'll touch on a few issues here with my two cents, but I feel like @Zepan 's comments were rather sufficient.

  • First, hard-coding dns servers IS bad practice, but its hardly out of the norm these days. Companies fed up with project's like Pi-Hole are starting to use this more and more frequently. I have even seen some dns-over-tls and https requests happen without my consent (from other devices, looking at you, Google.). So, I don't feel that call out is exactly fair.
  • The root password, ok, so it's root? I didn't buy this thing to manage a server running in production in some fortune 500 company. I also expect to be able to manage it, which I can. I'm more concerned over the weak ssh ciphersuites this thing supports, but it's not a powerful (or expensive) device.
  • Lack of TLS. Yep it lacks TLS, but you can enable that. The documentation even has instructions for you, but it does impact performance. Again, it's not a powerful (or expensive) device.
  • I agree the update method requires a bit of a re-think. I would like to see the sources open as well and signed at a minimum.
  • Tailscale, is installed and the daemon is running and checking in with the tailscale network. Yep, that's how that works! The odd thing here, it's not happening on my 5 devices. I suspect that is because I received my device very early, before the tailscale support was added. I have not seen this connectivity, but I also either disabled talescale or it was never enabled in the updates I have received. Tailescale is awesome and I would naturally be suspicious, but it does a lot on it's own to work, and this might not be the NanoKVM doing scary things, but tailscale. I have not seen any evidence that a tailescale network was added without my knowledge. I would rather see this disabled by default, as is my experience, but that seems to have changed during recent revisions.

In summary, yes, there are some security issues, outdated packages, weak ssh ciphers/hmac, poor update mechanism/validation to name a few. But it's a $20.00 KVM. I didn't expect the gold standard here, I trust it about as much any device, but I have seen little evidence that this is a "backdoor".

I want to say something about the timing of the release of a competing kvm product, but I have no evidence to even make an assumption. What I do want to see happen is Sipeed make good on their answers here and opensource the rest of the project, but I don't expect perpetual updates for a $20.00 KVM.

@That-Dude
Copy link

This is a KVM that is going to be connected to servers that are logged into admin / root sessions. There is no trade off with usability versus security, its 100% security first, usability second.

All of @apalrd points are valid. Your hardcoded keys, password management, unverified downloads from remote servers, dns bypassing and tailscale on by default are a security disaster waiting to happen.

Fix these issues and become the homelabbers product of choice, you'll quickly gain trust and can break into the massive SME market. If you don't do it a competitor will and you're going to miss out on a huge opportunity! I have 4 nanokvms and i think the hardware is great, but they live in a restricted vlan for the reasons listed above.

@rosshiga
Copy link

rosshiga commented Feb 6, 2025

I agree it's alot of criticism for a hobby level just out of beta product but the attitude that it's just going to be used in home lab is why crap IoT security exist. This is a remote access device. Secured by design is a philosophy and unless you change your mindset the product will never be secure enough for a RAT.

The glaring difference here between you and JetKVM is that when stuff like this comes up JetKVM Team says "no, user, you will hurt yourself" in the long term if you don't change your password. You should not be making the product more insecure. Convenience is at the expense of security and it's clear what you folks believe is more important.

@swaits
Copy link

swaits commented Feb 6, 2025

There is no reason (maybe other than ntp) a KVMoverIP device needs to reach out to the public Internet, ever. No good reason. At least not by default.

Any claim otherwise is in very bad faith.

Now, I'm going to assume you're not doing anything malicious. But, it doesn't change the need to come completely clean and fix it. Why? Hanlon's Razor:

Never attribute to malice that which is adequately explained by stupidity.

@bddanford
Copy link

Could we get the most current release of the firmware posted maybe? Some of the glaring issues can be quickly resolved. I would really like to see these devices work and have some reasonable expectation of a somewhat secured by default. If this stuff was mostly done to make PoC/testing easier, then thats fine, tell us, be up front, tell us what and why its doing it, and keep it a beta product until you are ready to lock it down more and move to more of a release candidate testing phase.

@jduncanator
Copy link

jduncanator commented Feb 7, 2025

Hardcoded Keys in TypeScript
Yes, this was an improper practice, a flaw left over from the early rapid development stage. We plan to move the keys into configuration files in the next version, which is expected to be released this month. This improvement will further enhance product security. However, it is worth noting that the primary security risk associated with this issue is man-in-the-middle attacks, meaning that the risk arises mainly when your network is untrusted or malicious. In such cases, the number of potentially compromised devices around you may be greater than expected.

Why move the keys to a config file? The problem here is the fact that the password is encrypted and can be decrypted with a key.

  1. The password and salt should be hashed by the browser using a one way hash function such as sha256.
  2. The hash is sent over the network
  3. The Server (NanoKVM) should compare the hash with the hash stored locally.
  4. If the hashes match authentication was successful.

Employing Symmetric-key cryptography such as AES256 for login functionality such as this is very suspicious as it allows a man in the middle to decrypt the password if they have the key.

Using a one way hash function does not allow a threat actor to recover the password unless they have a very VERY large GPU farm to crack the hash.

These are basic security fundamentals.

While I understand the sentiment of what you were going for in this reply (and I agree, a better authentication solution is needed), implementing it exactly as outlined here also has a significant security issue.

You should never hash a password client-side, and then have the server compare that hash against the stored hash to perform authentication. Should the database ever be compromised, and the hashes of passwords leaked, it is trivial for an attacker to simply "replay" your hashed password from the database against the server and authenticate as you. You should always hash the provided authentication string (ie. users password) on the server, and then compare that against the hash in the database. Hashing the password client side, and then sending that over the network, has zero security benefits compared to just sending the password in the first place, and contributes to a false sense of increased security when in fact you've introduced a serious vulnerability.

@TuningYourCode
Copy link

TuningYourCode commented Feb 7, 2025

I do not have a NanoKVM (yet?) but i downloaded the latest release and you can simply extract and then mount the image by sudo mount -o loop,offset=16777728,rw 20241120_NanoKVM_Rev1_3_0.img /mnt (well you need a linux system/vm) and you can edit files (e.g. remove the adding of dns servers or deletion of tailscale) and then just flash/write the modified image to your sd card.

Also there is quiet much dev stuff on the image like jquery in uncompressed and minified version, quiet many development libs etc. so if somebody wants to slim down there is quiet much potential :)

Keep in mind that even professional KVM solutions should not be connected to the internet (at least not exposed to it) i would also recommend for these devices. Even if there are no security holes this device might easily be DDoSed with it's limited resources.

I think the product is super nice for home labbers and there will be some kind of community which will provide improved images.
The only issue might be if the linux kernel etc. has parts which are not open sourced - i didn't check that but it looks promising that they already open sourced parts of their code which always helps :)

I will most likely order one to get my hands on image creation for this device.

PS: How to get to 16777728

sudo apt install fdisk xz-utils
unxz 20241120_NanoKVM_Rev1_3_0.img.xz
sudo fdisk -l 20241120_NanoKVM_Rev1_3_0.img

Now simply multiply the Start of the Linux Partition (32769) with Sektor size (512).

@sean-wilkinson
Copy link

Hardcoded Keys in TypeScript
Yes, this was an improper practice, a flaw left over from the early rapid development stage. We plan to move the keys into configuration files in the next version, which is expected to be released this month. This improvement will further enhance product security. However, it is worth noting that the primary security risk associated with this issue is man-in-the-middle attacks, meaning that the risk arises mainly when your network is untrusted or malicious. In such cases, the number of potentially compromised devices around you may be greater than expected.

Why move the keys to a config file? The problem here is the fact that the password is encrypted and can be decrypted with a key.

  1. The password and salt should be hashed by the browser using a one way hash function such as sha256.
  2. The hash is sent over the network
  3. The Server (NanoKVM) should compare the hash with the hash stored locally.
  4. If the hashes match authentication was successful.

Employing Symmetric-key cryptography such as AES256 for login functionality such as this is very suspicious as it allows a man in the middle to decrypt the password if they have the key.
Using a one way hash function does not allow a threat actor to recover the password unless they have a very VERY large GPU farm to crack the hash.
These are basic security fundamentals.

While I understand the sentiment of what you were going for in this reply (and I agree, a better authentication solution is needed), implementing it exactly as outlined here also has a significant security issue.

You should never hash a password client-side, and then have the server compare that hash against the stored hash to perform authentication. Should the database ever be compromised, and the hashes of passwords leaked, it is trivial for an attacker to simply "replay" your hashed password from the database against the server and authenticate as you. You should always hash the provided authentication string (ie. users password) on the server, and then compare that against the hash in the database. Hashing the password client side, and then sending that over the network, has zero security benefits compared to just sending the password in the first place, and contributes to a false sense of increased security when in fact you've introduced a serious vulnerability.

Yes I agree, the hash can be replayed but at least the original password can not be recovered which could potentially match the root/admin password of the host machine, as the OP stated and we all know users are bad with passwords.

This is why public-key cryptography exists, why TLS is important and should be on my default. I do understand there are performance implications in doing this thought.

As others have also stated security should be a top priority on a device such as this, the community and myself as a customer are not going to take the product seriously unless the software can be trusted and is secure.

@Zepan
Copy link
Contributor Author

Zepan commented Feb 7, 2025

Dear Sipeed Team,

I assume that due to the recently released video, you will now receive increased feedback as well as critical messages—some of which are justified. However, I am convinced that you can use this situation as a great opportunity to become even more successful in the long term.

My advice: Support the open source community by releasing all the information needed to build a clean and efficient open source firmware. You will see that many more people will be willing to purchase your hardware because that is your unique selling point!

Personally, I would have bought the nanoKVMs much earlier if the open source aspect had been clearly resolved and everything had been made open source. Don’t be afraid to build something great together with the community—and don’t fear competition. If you deliver top-notch hardware while quickly and sustainably fixing the software issues, your project will be celebrated and widely adopted.

I truly appreciate the price/performance ratio of your hardware—even compared to the JetKVM. Now it’s up to you to optimize the software and release it fully open source. Be smart and let the community help you.

Best regards!

Yes, we have update roadmap in readme, the buildable SDK will totally availlable in Q1, before new NanoKVM-Pro release.

@wj-xiao
Copy link
Collaborator

wj-xiao commented Feb 7, 2025

In the past few months, we have been focusing on product features and user experience, and have ignored the security issues.

We simply assumed that users could rely on Tailscale to mitigate security risks when exposing their devices to the public internet. This was a mistake.

We will fix these security vulnerabilities, and a new version will be released this month.

@swaits
Copy link

swaits commented Feb 7, 2025

It's GPL. Someone should just fork this and create a community version. I don't have the bandwidth at the moment. But might in a few months.

@jduncanator
Copy link

It's GPL. Someone should just fork this and create a community version. I don't have the bandwidth at the moment. But might in a few months.

Nearly all of the stated security issues are in the firmware/OS image, and not the user space application that is open-source.

From my understanding, the OS build sources/process and kernel tree are not open-source, and so it's impossible to build a new firmware image from scratch even if you wanted to at the moment.

I agree though, Sipeed should prioritise open-sourcing the distribution and firmware source so contributors can help fix some of these issues.

@markuman
Copy link

markuman commented Feb 7, 2025

We will remove the modification operation and no longer modify the DNS configuration file.

For the record. The nanokvm does, once it's booted, just a few dns requests for some ntp.org domains. That's it. No more dns queries in the last 24h here (recorded with adguard home).
When it searches for updates, it asks for a google domain, a cloudflare domain and a sipeed domain.
So claiming that the devices is spyware is currently conspiracy theory.
And that the devices is calling home (which is obversley china, for a chinese company) to ask for updates, is also not a surprise.

But don't get me wrong. I don't trust China any more than I trust the US or any other government or "cloud" company.

Yes, there are some security concerns or something that is personal annoying (tailscaled when you don't use tailscale). I removed it initially after it arrives here in early december and the system load goes below 4 from 6. That alone is a reason for me to remove it.

But in the current point of time, it feels like the entire internet is overreacting.
And when someone exposes any device (without reviewing or firewalling) plain to the internet, it's not the fault of the vendor.

update 12h ago from initial comment

For the record. The nanokvm does, once it's booted, just a few dns requests for some ntp.org domains. That's it. No more dns queries in the last 24h here (recorded with adguard home).

This is only true if tailscaled is stopped and disabled.

@Freekers
Copy link

Freekers commented Feb 7, 2025

We will remove the modification operation and no longer modify the DNS configuration file.

For the record. The nanokvm does, once it's booted, just a few dns requests for some ntp.org domains. That's it. No more dns queries in the last 24h here (recorded with adguard home).

Unless you are redirecting all DNS queries on port 53 to your AdGuard instance using a firewall rule in your router, you won’t see them. This is because, as mentioned before, NanoKVM continuously changes or adjusts its DNS servers to Google DNS. NanoKVM ignores any DNS servers set via DHCP after the initial boot.

@markuman
Copy link

markuman commented Feb 7, 2025

We will remove the modification operation and no longer modify the DNS configuration file.

For the record. The nanokvm does, once it's booted, just a few dns requests for some ntp.org domains. That's it. No more dns queries in the last 24h here (recorded with adguard home).

Unless you are redirecting all DNS queries on port 53 to your AdGuard instance using a firewall rule in your router, you won’t see them. This is because, as mentioned before, NanoKVM continuously changes or adjusts its DNS servers to Google DNS. NanoKVM ignores any DNS servers set via DHCP after the initial boot.

Yes of course I've read, watch, and noticed this all - which does not mean that is not unfixable.
And I've fixed that on my own permanently. See https://github.com/markuman/nanokvm-mitigations/blob/latest/cleanup.yml#L24-L31 with using dns: 192.168.178.50, which is my adguard home dns server - that is also distributed via dhcp in my network.

So my /etc/resolv.conf is immutable and I see all dns queries that the nanokvm is doing.

@apalrd
Copy link

apalrd commented Feb 7, 2025

A packet capture from boot also shows log.tailscale.io, stun.l.google.com, and turn.cloudflare.com, a NAT-PMP port map request, and HTTPS and STUN traffic, even when it has never been configured to do any sort of cloud connectivity. So it's not just pool.ntp.org. Obviously we know all of this is Tailscale, but an out of the box unconfigured device trying to poke holes in the firewall is absolutely not okay.

As to calling home, going to a CDN to download an update is normal, yes, but calling to a server with your serial number to get a customized binary which can't be verified to be correct is another thing entirely. Since the binaries are customized, you have no way of verifying that your binary doesn't have something malicious in it, since you can't verify the hash of the download matches what everyone else is getting. Even though nobody suspects Sipeed is doing that, the process is designed so badly that it's a clear possibility.

@markuman
Copy link

markuman commented Feb 7, 2025

So it's not just pool.ntp.org.

Yes, if you keep it configured with tailscaled, than its more than the ntp.org request. True.
And the stun requests to google and cloudflare is also triggered by the request for updates, even if tailscaled is turned of.

Since the binaries are customized, you have no way of verifying that your binary doesn't have something malicious, ...

That's the nature of proparitary/closed software/firmware - like you get with your iPhone/Android, Fritzbox Router, Windows Notebook, Unifi Router ...

... since you can't verify the hash of the download matches what everyone else is getting.

To be honest, a hash would just safe you for a corrupt download file. 😀
To make it safe, sipeed must use dnssec, that nanokvm must validate and the firmware must be signed with gpg.

@BlwAvg
Copy link

BlwAvg commented Feb 7, 2025

ab3d980#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5

They have made a roadmap with upcoming changes

@apalrd
Copy link

apalrd commented Feb 8, 2025

That's the nature of proparitary/closed software/firmware - like you get with your iPhone/Android, Fritzbox Router, Windows Notebook, Unifi Router ...

Except you can normally compare your closed-source binary files to everyone else's to see if yours has been tampered with. Just because it's closed-source doesn't mean it can't be signed and the binary hashes can't be listed publicly.

@byrevx
Copy link

byrevx commented Feb 8, 2025

/kvmapp/system/init.d/S30eth

nameserver $gw
nameserver 8.8.8.8
nameserver 114.114.114.114

Regarding these servers, yes, I think the two (Secondary or Tertiary DNS) should not exist. Any network must have a DNS, otherwise what's the point of the network to access anything through named domains? Most of us have DNS provided via DHCP, we don't bother with it too much.

The first is from Google. The second is a Chinese public DNS and is widely used in China and offers an alternative to local ISP-provided DNS services.
It was correct that the products intended for the Chinese market should have Chinese DNS and the rest with Secondary or Tertiary DNS from Google 4.4.4.4 or 8.8.8.8 and Cloudflare: 1.1.1.2 or 1.0.0.3 with malware blocking.

From what I have observed, the configuration is saved in /etc/resolve.conf , so it can practically be changed at any time, it is not written in a PROM.
Apparently there have probably been other changes, this is the first time I've looked through the code out of curiosity.

$gw is basically the network gateway that is provided or calculated. From what I can see it is initially read from /boot/eth.nodhcp and the file suggests that DHCP is not used ... then where did the gateway come from, strange.

I was going to buy myself two PCIe models, so I still don't have something to experiment with, to figure out how much "open source" or "proprietary" it is!

And as someone else said in the comments: Dear Sipeed, "don't miss this train", this year some models based on this project will definitely start appearing and if they are made "flawlessly" you have already lost a very, very big opportunity, I have the impression that you don't even realize the magnitude of it!

P.S. Maybe I got involved without being asked, and I didn't understand the code on github well at first glance!

@gitautas
Copy link

gitautas commented Feb 8, 2025

I feel as though there is a heavy misunderstanding for why the device was made and the reality of product development. This is a product made for hobbyist use, it is priced accordingly and seems to me is made with a lot of good will. Just between the comments on this issue and the video in question, it seems to me as though few people have expertise in embedded webRTC on riscv and the realities that come with that, especially when making a product that should "just work" in as many environments as possible. The reasoning given makes sense to me, these are choices and mistakes that I can entirely see myself making if I were in their place.

I want to give thanks to the Sipeed team for creating an open product in the first place and that meets my expectations 100%.

@ddscentral
Copy link

Partially agree. But some of the noted bugs were easily avoidable early in the development process.

For example the hard-coded password inside JS and use of AES256 instead of industry standard hashes. Or use of hard-coded DNS servers instead of following industry standard of using DNS supplied by DHCP. And the tailscale running all the time, phoning home and opening ports even when it was not enabled by the user.
Not sure whey they haven even included a VPN client, considering this chip is really underpowered CPU-wise and can barely handle a VPN.

Indeed what they did for the price is pretty amazing. But still KVM is a really sensitive type of device which has direct access to your server and your data. If you can't trust it, you simply should not use it.

@alexisfrjp
Copy link

All these people complaining, so ridiculous.

How many closed-source devices you have in your network, starting with your provider's router.
99.9% of NanoKVM users are still on windows 10/11 with virus already installed.

Please don't make the life of 99.9% more complicated for the 0.1% pedantic users.
Nobody prevents crybabies from disabling ssh, removing tailscale and making it secure as they like.
Pathetic.

@Alex4386
Copy link

Alex4386 commented Feb 10, 2025

All these people complaining, so ridiculous.

How many closed-source devices you have in your network, starting with your provider's router. 99.9% of NanoKVM users are still on windows 10/11 with virus already installed.

That's BS. You DON'T want more virus and reducing risks are good thing.
It is getting people's attention since it is an opensource project. and that's good thing, making NanoKVM more secure. If they are doing something nefarious, You know how to remove it.

Please don't make the life of 99.9% more complicated for the 0.1% pedantic users. Nobody prevents crybabies from disabling ssh, removing tailscale and making it secure as they like. Pathetic.

Nah, I really don't think so since I have bought NanoKVM on the initial release, at that time the code wasn't even open source, "web" (a.k.a. "web interface") was the only directory available on this repo and on their site (SiPeed's wiki page), they had some "Star Me" campaign to get a server codebase.
Considering this. Yes, They did prevent people from removing ssh and other stuff on their system, "cause it WAS NOT opensource". (See #1), and I don't want something managing critical infrastructure without either binding contract (commercial iKVMs with the server vendor's warranty) or open-source system that can personally audit into.

Unfortunately, due to this, I wasn't able to "trust" NanoKVM for the deployment back when I got my stuff, although this wasn't the only reason (e.g. NanoKVM getting finicky on some weird resolutions that some of the servers do, can't remember the details so I can not escalate as an issue for now)

@gshpychka
Copy link

In the past few months, we have been focusing on product features and user experience, and have ignored the security issues.

This is pretty wild for a KVM, to be honest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests