Maintenance of pandas type stubs

[Dr-Irv]

The pandas team has an open issue with regards to type stubs: pandas-dev/pandas#28142

We'd like to have some discussion with the key developers (@jakebailey ? @gramster ? others ?) of the pandas type stubs to determine a best way forward for upkeep of these type stubs. I brought this up at our December, 2020, monthly core developers meeting.

Might be best for you guys to contribute over there in that pandas issue.

Thanks!

[gramster]

Hi Irv, we'd be very interested in having this discussion; what forum do you suggest?

[Dr-Irv]

Hi Irv, we'd be very interested in having this discussion; what forum do you suggest?

Thanks for commenting on the pandas issue. I will also bring it up with the core team as we may want to do a deeper discussion in our next core dev meeting (which I believe is on 1/13/2021).

[Dr-Irv]

@gramster @erictraut Info on our monthly dev meeting can be found here: https://pandas.pydata.org/docs/development/meeting.html

It would be great if you could attend the next one on 1/13/2021 at 1PM Eastern time, as we'd like to have a discussion with you on how to move forward. Thanks!

[gvanrossum]

I'd like to attend -- is that okay? I might know a thing or two about stubs (though not necessarily about the ins and outs of how Pylance and Pyright use them -- the expert there would be @erictraut).

[Dr-Irv]

@gvanrossum absolutely! Our pandas dev meetings are open to all. When I get more info, I will post here.

[Dr-Irv]

@gramster @erictraut @gvanrossum Info on tomorrow's dev meeting can be found here:
pandas-dev/pandas#39125

[gramster]

I think we have a rough plan of action:

• add tests to the stubs to improve the quality and avoid regressions. The adding of initial tests has been done (b19436a) although there is work to be done to pass them and we'll likely want more;
• following that, we can start upstreaming some of the type annotations from here into pandas; my hope is that we can do that on a file-by-file basis, and as we complete each file, we can remove it from here and start regenerating the stub from the pandas source. I still need to add the tooling to do the latter but will do that once we get to the initial set of upstream commits.

[gvanrossum]

How do our stubs compare to this? https://pypi.org/project/pandas-stubs/

[Dr-Irv]

@gvanrossum The tests that @gramster has added came from that project. The pandas team had a meeting with @gramster and the team behind https://pypi.org/project/pandas-stubs/ and we decided on a plan of action to get stubs into the pandas project itself. At that meeting, we decided that the stubs here were more complete than those, but they have a great testing mechanism, which we are adopting here, and eventually in the pandas project.

[gvanrossum]

Ah, great. Never mind me then!

[gramster]

Moving this to discussions.

[Dr-Irv]

@gramster I see that you added tests corresponding to the issues. But for many of the things I fixed, which I think started with #64 and later, I just put the "failing" code in the PR, and didn't create an issue. So we should pull in more regressions from all those small examples.

I'll soon be doing some new PRs related to a few issues found in one of our codes that we recently converted to have better typing. For each of them, I'll include a test script now that the framework is all set up.