Document upload API so other projects can implement it

[takluyver]

Is your feature request related to a problem? Please describe.

For Python projects, the new uploader is a big step backwards in usability, except on the CI platforms where it's pre-packaged (like Github actions).

The deprecated Python uploader is automatically cross platform and cross architecture, and available as a 20 kB download from PyPI, working with the tools and security model we're used to (the security model could be stronger, admittedly, but it's something we rely on anyway). The new uploader is available as 4 different platform-specific binaries, doesn't support all architectures we want to use (e.g. codecov/uploader#523), it's a ~40 MB download, and we either have to fiddle around with GPG or trust your download site (though I'm sure you take the security of that very seriously).

Describe the solution you'd like

I appreciate that codecov doesn't want to maintain uploaders in N different languages. But would it be possible to document the HTTP requests involved in uploading coverage data, and let communities maintain upload tools on their own?

• There's already at least one third-party uploader tool for Python which has presumably reverse-engineered what it needs, so there's definitely an appetite for this: https://github.com/seantis/pytest-codecov
• The deprecated Python uploader still works, and appears to use the same /upload/v4/ URLs as the typescript code, so it looks like API churn is not a major issue.

Describe alternatives you've considered

• Using the new uploader. The biggest issue for us is the lack of binaries for some architectures we're testing on - the other things are just annoyances.
• Sticking with the old Python uploader. This is what we're doing for now. It didn't break in February (as the deprecation plan suggested), but I assume you will either deliberately disable it at some point, or incidentally break it when you change the API.
• Reverse-engineering the HTTP requests and imitating the new uploader to pass the server-side checks. I'd rather not make this adversarial. 🙂

[drazisil-codecov]

@takluyver Thank you for the detailed issue!

Do the methods in https://github.com/codecov/uploader/blob/84df6d138c97db110d73345a74f97bca9a904d96/src/helpers/web.ts (the only part of the uploader that interacts with the upload endpoint in any way), meet your needs, or do you need something more detailed/formal?

[takluyver]

Thanks! I can figure out roughly what it's doing; I guess I'm asking for documentation less for information, and more as a kind of indication of stability & availability. If I use something other than the official typescript uploader code, I'd like to have some expectation that it will keep working, and some channel for anyone maintaining alternate implementations to hear about changes.

In the first instance, do you know what's happening with the 'sunsetting' of other uploaders? The deprecation plan laid out that they would stop working in February, which didn't happen (at least for the Python uploader). Is that off, or just delayed? If it's still happening, will it be by checking the package= URL parameter, and will it block everything apart from the new uploader, or just specific old uploaders?

[drazisil-codecov]

!!! The answer below is my personal thoughts and does not reflect the official or unofficial stance of Codecov !!!

In the first instance, do you know what's happening with the 'sunsetting' of other uploaders? The deprecation plan laid out that they would stop working in February, which didn't happen (at least for the Python uploader). Is that off, or just delayed? If it's still happening, will it be by checking the package= URL parameter, and will it block everything apart from the new uploader, or just specific old uploaders?

It is delayed, due to the new uploader not being quite in parity with the old ones yet. I'm not sure the exact mechanism being employed, but the goal is to have a single source of truth when it comes to uploading, rather than several different ones in the wild with expectations of security and maintenance around them.

indication of stability & availability. If I use something other than the official typescript uploader code, I'd like to have some expectation that it will keep working, and some channel for anyone maintaining alternate implementations to hear about changes.

I can't speak to if we want or plan to make the uploader endpoint a documented stable endpoint, for the reasons outlined above. What if a custom uploader starts uploading source code, or modifying files? Who would be considered responsible for that, from a legal and reasonable standpoint?

I'm neither legal nor product, so I think I should probably step away so as not to set possibly incorrect expectations. :)

!!! The answer above is my personal thoughts and does not reflect the official or unofficial stance of Codecov !!!

[drazisil-codecov]

Officially,

Moved to a feature request (currently private)
https://codecov.canny.io/feedback/p/document-upload-api-so-other-projects-can-implement-it
@codecov/product

[takluyver]

Thanks @drazisil-codecov

What if a custom uploader starts uploading source code, or modifying files? Who would be considered responsible for that, from a legal and reasonable standpoint?

I'm very much not a lawyer, but it's not unusual for a company to provide a REST API (e.g. Github's API) and other people entirely to provide tools that use that API (e.g. github3.py). I think it's pretty clear to everyone involved that you're trusting whoever wrote the wrapper or tool you're using, not just the company whose API it uses, and you can't blame the company if a third-party wrapper turns malicious.

I understand you'd want to be clear about the boundaries of what's maintained by codecov and what's not, and that's somewhat more challenging because you used to maintain lots of uploader tools. But I hope that's not an insurmountable obstacle. 🙂

[adiroiban]

+1 for getting the API published.

The canny.io website only allows Google accounts. For codecov I am using my github account.

I am still using a python uploaded (from a local mirror) and it works.

[thomasrockhu-codecov]

As it stands, we currently don't have plans to add documentation here as we are constantly making changes to improve reliability.

Since many of these changes can be breaking, we make efforts to ensure that the changes are reflected in our upload tools. As a result, these is not a good candidate for documentation currently.

cc: @rohan-at-sentry