-
Notifications
You must be signed in to change notification settings - Fork 100
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Response to concerns about NanoKVM security #301
Comments
When you say "next version", are you talking about a new hardware (NanoKVM Pro), or a software update for current NanoKVM users? |
This feels really dismissive. Let's take the hardcoded DNS servers as an example, but this applies to multiple issues responded to here. Hardcoded DNS is a pretty obvious bad practice. Your reply is: "Some users in certain regions experience domain resolution failures with their local DNS providers." Then why not have a sane default and offer those few users who need to take extra steps the ability to easily change the DNS server? Just because users in China are behind a firewall, doesn't mean that by default this device should ignore the DNS server in my network. |
The software update. |
We will remove the modification operation and no longer modify the DNS configuration file. |
Why move the keys to a config file? The problem here is the fact that the password is encrypted and can be decrypted with a key.
Employing Symmetric-key cryptography such as AES256 for login functionality such as this is very suspicious as it allows a man in the middle to decrypt the password if they have the key. Using a one way hash function does not allow a threat actor to recover the password unless they have a very VERY large GPU farm to crack the hash. These are basic security fundamentals. |
Dear Sipeed Team, I assume that due to the recently released video, you will now receive increased feedback as well as critical messages—some of which are justified. However, I am convinced that you can use this situation as a great opportunity to become even more successful in the long term. My advice: Support the open source community by releasing all the information needed to build a clean and efficient open source firmware. You will see that many more people will be willing to purchase your hardware because that is your unique selling point! Personally, I would have bought the nanoKVMs much earlier if the open source aspect had been clearly resolved and everything had been made open source. Don’t be afraid to build something great together with the community—and don’t fear competition. If you deliver top-notch hardware while quickly and sustainably fixing the software issues, your project will be celebrated and widely adopted. I truly appreciate the price/performance ratio of your hardware—even compared to the JetKVM. Now it’s up to you to optimize the software and release it fully open source. Be smart and let the community help you. Best regards! |
I personally find apalrd’s security concerns to be perfectly valid, given what type of device this is. This is a KVM device which is used to REMOTELY CONTROL a computer. I have acquired several NanoKVM devices but so far I only put one of them in service for testing/tinkering purposes. |
Why don't you respond to the specific issues in this repo, some of which are about the concerns you're addressing? |
I didn't pull my comments out of thin air. I read all of the Github issues that have been accumulating here for months before making the video, and covered the ones I thought were the most significant. The lack of action on these issues (here, on Github) speaks volumes to your commitments to security in the product. |
Honestly, I don't think the review by apalrd's video was fair. I knew going in that this device was from a possible suspect source, in reality or not, but let's be clear everything on my network is a suspect. So, I treat it that way. Using sensationalized words and phrases like "Backdoor" and "Victim's computer" seems a bit unfair when there was no evidence to indicate any of that was true. I was suspect of these and isolated them, but, honestly, Google devices make more scary calls than this thing does. I'll touch on a few issues here with my two cents, but I feel like @Zepan 's comments were rather sufficient.
In summary, yes, there are some security issues, outdated packages, weak ssh ciphers/hmac, poor update mechanism/validation to name a few. But it's a $20.00 KVM. I didn't expect the gold standard here, I trust it about as much any device, but I have seen little evidence that this is a "backdoor". I want to say something about the timing of the release of a competing kvm product, but I have no evidence to even make an assumption. What I do want to see happen is Sipeed make good on their answers here and opensource the rest of the project, but I don't expect perpetual updates for a $20.00 KVM. |
This is a KVM that is going to be connected to servers that are logged into admin / root sessions. There is no trade off with usability versus security, its 100% security first, usability second. All of @apalrd points are valid. Your hardcoded keys, password management, unverified downloads from remote servers, dns bypassing and tailscale on by default are a security disaster waiting to happen. Fix these issues and become the homelabbers product of choice, you'll quickly gain trust and can break into the massive SME market. If you don't do it a competitor will and you're going to miss out on a huge opportunity! I have 4 nanokvms and i think the hardware is great, but they live in a restricted vlan for the reasons listed above. |
I agree it's alot of criticism for a hobby level just out of beta product but the attitude that it's just going to be used in home lab is why crap IoT security exist. This is a remote access device. Secured by design is a philosophy and unless you change your mindset the product will never be secure enough for a RAT. The glaring difference here between you and JetKVM is that when stuff like this comes up JetKVM Team says "no, user, you will hurt yourself" in the long term if you don't change your password. You should not be making the product more insecure. Convenience is at the expense of security and it's clear what you folks believe is more important. |
There is no reason (maybe other than ntp) a KVMoverIP device needs to reach out to the public Internet, ever. No good reason. At least not by default. Any claim otherwise is in very bad faith. Now, I'm going to assume you're not doing anything malicious. But, it doesn't change the need to come completely clean and fix it. Why? Hanlon's Razor:
|
Could we get the most current release of the firmware posted maybe? Some of the glaring issues can be quickly resolved. I would really like to see these devices work and have some reasonable expectation of a somewhat secured by default. If this stuff was mostly done to make PoC/testing easier, then thats fine, tell us, be up front, tell us what and why its doing it, and keep it a beta product until you are ready to lock it down more and move to more of a release candidate testing phase. |
While I understand the sentiment of what you were going for in this reply (and I agree, a better authentication solution is needed), implementing it exactly as outlined here also has a significant security issue. You should never hash a password client-side, and then have the server compare that hash against the stored hash to perform authentication. Should the database ever be compromised, and the hashes of passwords leaked, it is trivial for an attacker to simply "replay" your hashed password from the database against the server and authenticate as you. You should always hash the provided authentication string (ie. users password) on the server, and then compare that against the hash in the database. Hashing the password client side, and then sending that over the network, has zero security benefits compared to just sending the password in the first place, and contributes to a false sense of increased security when in fact you've introduced a serious vulnerability. |
I do not have a NanoKVM (yet?) but i downloaded the latest release and you can simply extract and then mount the image by Also there is quiet much dev stuff on the image like jquery in uncompressed and minified version, quiet many development libs etc. so if somebody wants to slim down there is quiet much potential :) Keep in mind that even professional KVM solutions should not be connected to the internet (at least not exposed to it) i would also recommend for these devices. Even if there are no security holes this device might easily be DDoSed with it's limited resources. I think the product is super nice for home labbers and there will be some kind of community which will provide improved images. I will most likely order one to get my hands on image creation for this device. PS: How to get to
Now simply multiply the Start of the Linux Partition (32769) with Sektor size (512). |
Yes I agree, the hash can be replayed but at least the original password can not be recovered which could potentially match the root/admin password of the host machine, as the OP stated and we all know users are bad with passwords. This is why public-key cryptography exists, why TLS is important and should be on my default. I do understand there are performance implications in doing this thought. As others have also stated security should be a top priority on a device such as this, the community and myself as a customer are not going to take the product seriously unless the software can be trusted and is secure. |
Yes, we have update roadmap in readme, the buildable SDK will totally availlable in Q1, before new NanoKVM-Pro release. |
In the past few months, we have been focusing on product features and user experience, and have ignored the security issues. We simply assumed that users could rely on Tailscale to mitigate security risks when exposing their devices to the public internet. This was a mistake. We will fix these security vulnerabilities, and a new version will be released this month. |
It's GPL. Someone should just fork this and create a community version. I don't have the bandwidth at the moment. But might in a few months. |
Nearly all of the stated security issues are in the firmware/OS image, and not the user space application that is open-source. From my understanding, the OS build sources/process and kernel tree are not open-source, and so it's impossible to build a new firmware image from scratch even if you wanted to at the moment. I agree though, Sipeed should prioritise open-sourcing the distribution and firmware source so contributors can help fix some of these issues. |
For the record. The nanokvm does, once it's booted, just a few dns requests for some ntp.org domains. That's it. No more dns queries in the last 24h here (recorded with adguard home). But don't get me wrong. I don't trust China any more than I trust the US or any other government or "cloud" company. Yes, there are some security concerns or something that is personal annoying (tailscaled when you don't use tailscale). I removed it initially after it arrives here in early december and the system load goes below 4 from 6. That alone is a reason for me to remove it. But in the current point of time, it feels like the entire internet is overreacting. update 12h ago from initial comment
This is only true if tailscaled is stopped and disabled. |
Unless you are redirecting all DNS queries on port 53 to your AdGuard instance using a firewall rule in your router, you won’t see them. This is because, as mentioned before, NanoKVM continuously changes or adjusts its DNS servers to Google DNS. NanoKVM ignores any DNS servers set via DHCP after the initial boot. |
Yes of course I've read, watch, and noticed this all - which does not mean that is not unfixable. So my |
A packet capture from boot also shows log.tailscale.io, stun.l.google.com, and turn.cloudflare.com, a NAT-PMP port map request, and HTTPS and STUN traffic, even when it has never been configured to do any sort of cloud connectivity. So it's not just pool.ntp.org. Obviously we know all of this is Tailscale, but an out of the box unconfigured device trying to poke holes in the firewall is absolutely not okay. As to calling home, going to a CDN to download an update is normal, yes, but calling to a server with your serial number to get a customized binary which can't be verified to be correct is another thing entirely. Since the binaries are customized, you have no way of verifying that your binary doesn't have something malicious in it, since you can't verify the hash of the download matches what everyone else is getting. Even though nobody suspects Sipeed is doing that, the process is designed so badly that it's a clear possibility. |
Yes, if you keep it configured with tailscaled, than its more than the ntp.org request. True.
That's the nature of proparitary/closed software/firmware - like you get with your iPhone/Android, Fritzbox Router, Windows Notebook, Unifi Router ...
To be honest, a hash would just safe you for a corrupt download file. 😀 |
ab3d980#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5 They have made a roadmap with upcoming changes |
Except you can normally compare your closed-source binary files to everyone else's to see if yours has been tampered with. Just because it's closed-source doesn't mean it can't be signed and the binary hashes can't be listed publicly. |
/kvmapp/system/init.d/S30eth
Regarding these servers, yes, I think the two (Secondary or Tertiary DNS) should not exist. Any network must have a DNS, otherwise what's the point of the network to access anything through named domains? Most of us have DNS provided via DHCP, we don't bother with it too much. The first is from Google. The second is a Chinese public DNS and is widely used in China and offers an alternative to local ISP-provided DNS services. From what I have observed, the configuration is saved in /etc/resolve.conf , so it can practically be changed at any time, it is not written in a PROM. $gw is basically the network gateway that is provided or calculated. From what I can see it is initially read from /boot/eth.nodhcp and the file suggests that DHCP is not used ... then where did the gateway come from, strange. I was going to buy myself two PCIe models, so I still don't have something to experiment with, to figure out how much "open source" or "proprietary" it is! And as someone else said in the comments: Dear Sipeed, "don't miss this train", this year some models based on this project will definitely start appearing and if they are made "flawlessly" you have already lost a very, very big opportunity, I have the impression that you don't even realize the magnitude of it! P.S. Maybe I got involved without being asked, and I didn't understand the code on github well at first glance! |
I feel as though there is a heavy misunderstanding for why the device was made and the reality of product development. This is a product made for hobbyist use, it is priced accordingly and seems to me is made with a lot of good will. Just between the comments on this issue and the video in question, it seems to me as though few people have expertise in embedded webRTC on riscv and the realities that come with that, especially when making a product that should "just work" in as many environments as possible. The reasoning given makes sense to me, these are choices and mistakes that I can entirely see myself making if I were in their place. I want to give thanks to the Sipeed team for creating an open product in the first place and that meets my expectations 100%. |
Partially agree. But some of the noted bugs were easily avoidable early in the development process. For example the hard-coded password inside JS and use of AES256 instead of industry standard hashes. Or use of hard-coded DNS servers instead of following industry standard of using DNS supplied by DHCP. And the tailscale running all the time, phoning home and opening ports even when it was not enabled by the user. Indeed what they did for the price is pretty amazing. But still KVM is a really sensitive type of device which has direct access to your server and your data. If you can't trust it, you simply should not use it. |
All these people complaining, so ridiculous. How many closed-source devices you have in your network, starting with your provider's router. Please don't make the life of 99.9% more complicated for the 0.1% pedantic users. |
That's BS. You DON'T want more virus and reducing risks are good thing.
Nah, I really don't think so since I have bought NanoKVM on the initial release, at that time the code wasn't even open source, "web" (a.k.a. "web interface") was the only directory available on this repo and on their site (SiPeed's wiki page), they had some "Star Me" campaign to get a server codebase. Unfortunately, due to this, I wasn't able to "trust" NanoKVM for the deployment back when I got my stuff, although this wasn't the only reason (e.g. NanoKVM getting finicky on some weird resolutions that some of the servers do, can't remember the details so I can not escalate as an issue for now) |
This is pretty wild for a KVM, to be honest. |
In apalrd's recent video (https://www.youtube.com/watch?v=plJGZQ35Q6I), apalrd raised several security concerns regarding NanoKVM.
We have carefully watched the video—some parts are fair, while others result from an incomplete understanding of the product.
To help users better understand these concerns, we are providing a unified response under this issue. We welcome objective discussions.
Security vs. Usability Trade-off
Before discussing the details, we want to clarify a fundamental fact: security and usability are inherently contradictory to some extent. If one pursues absolute security, usability will be significantly compromised. apalrd represents a group of users who prioritize security to the extreme. However, based on our customer support records, a large proportion of users care more about ease of use.
For example, we receive multiple emails daily from users who have forgotten their self-set passwords and need help recovering them. apalrd also pointed out that our page "reminds" users to change their passwords rather than "forcing" them to do so, considering this a security risk.
In fact, we did consider forcing password changes during the initial product design. However, after shipping the alpha version, we found that a significant proportion (perhaps over 50%) of our users are beginners. If we force them to change their passwords, they are likely to forget them. Additionally, those who forget their passwords typically do not read the wiki (which contains instructions for physical factory reset). This would leave them unable to use the product or require them to send support emails.
Thus, we chose to remind rather than force users to change their passwords. apalrd mentioned that many IoT devices have similar security issues, but aside from actual security risks, some design choices are compromises for usability.
Responses to Other Concerns Before Discussing Security Details
We understand apalrd’s strong focus on security. However, as a product designed for a wide range of users, we must balance security and usability. We believe different user groups have different needs, so we made trade-offs in our product design accordingly. If apalrd had carefully read the wiki before using the product, some misunderstandings in the video could have been avoided.
Responses to Non-Security Design Issues
Response to Security Concerns
To address the security-related concerns, I have categorized all the issues and will respond accordingly.
Design Trade-offs for Usability and Availability
-As mentioned earlier, we chose to "remind" users to change their passwords rather than "force" them to do so.
Non-Security Issues / Other Feature Enhancements
Security Concerns Stemming from Tailscale or Network Configuration
Beneficial Security Recommendations
Potential Issues with SSH Enabled by Default
Initially, NanoKVM was designed as a product for homelab developers. Therefore, we set the product to "Developer Mode" by default, enabling SSH to facilitate network diagnostics and recovery for developers. Many of our previous designs and configurations, as well as our earlier products like SBCs and AI Cameras, were also tailored for developer users. However, as NanoKVM reaches more general users, it is indeed time to default to a "Regular User" mode, with "Developer Mode" requiring manual activation.
Hardcoded Keys in TypeScript
Yes, this was an improper practice, a flaw left over from the early rapid development stage. We plan to move the keys into configuration files in the next version, which is expected to be released this month. This improvement will further enhance product security. However, it is worth noting that the primary security risk associated with this issue is man-in-the-middle attacks, meaning that the risk arises mainly when your network is untrusted or malicious. In such cases, the number of potentially compromised devices around you may be greater than expected.
Lack of APP Update Verification
Similar to the previous issue, we will add verification in the next version. The primary risk also comes from man-in-the-middle attacks.
Summary
We are always committed to finding the best balance between security and usability and appreciate the feedback from apalrd and other users.
We will continue improving NanoKVM to ensure it meets the security needs of advanced users while remaining simple and user-friendly for beginners.
We welcome all users to transparently discuss and provide suggestions in an open-source environment. Whether the concerns are genuine security issues or security concerns influenced by bias, they can be properly addressed in a transparent, open discussion.
Additionally, we recommend users read our Wiki documentation before using NanoKVM to fully understand its features and usage.
For any questions, our technical support team is always available: [email protected].
The text was updated successfully, but these errors were encountered: