R Spatial data Working group

[basille]

I have sent the email below to a number of movement experts — it was suggested that we could have the discussion here instead, which I am happy to do. Let me start by simply posting my initial message here, followed by some follow-up and much needed clarification:

Dear friends and colleagues,

I hope this email finds you well and that you are all staying safe in the
midst of this crisis. With this email, I would like to suggest the creation
of an R Spatial data Working group (SWG) within the R Consortium, and
invite you to join us in the group.

This idea stems from our own experience (notably with the sftrack package
for tracking data), and discussions with Hadley Wickham (for the R
Consortium), which indicated that there does not seem to be shared
governance for spatial data in R. Following our own efforts in the realm of
movement, we would like to create such a collaborative group, to highlight
the strengths and weaknesses of R for spatial data, outline gaps and future
directions for development, and ultimately ensure that our efforts have a
maximum impact. Working groups within the R Consortium are very loosely
defined: all it takes is a declaration, and then it's up to us to formalize
how we want to move forward. A platform with a discussion forum and regular
online meetings (maybe twice a year) would be a start, and should be enough
to identify what is necessary to succeed.

Many of you have been active to different degrees in the spatial world in
the past years, especially as tracking/movement experts (including in other
environments than R, which is also very valuable). Our initial focus is
thus towards tracking data, but we would like to quickly scale it up to
spatial data in general (and some of you have expertise that reaches far
beyond movement strictly speaking). Consider the focus on tracking data as
a starting point, as the process is already ongoing — we will expand to all
spatial data in follow-up discussions, and invite other relevant experts.

My question at this stage is thus simple: Would you like to be part of this
conversation, and be part of the SWG? As the group will be rather informal,
there is no strong commitment expected, and instead this should come
naturally as part as your current efforts in the spatial world. Think of it
as a platform that primarily facilitates discussions about spatial data in
R for further development.

Best regards,
Mathieu Basille.

[basille]

To what @edzer answered (reproduced here with his permission):

Hi Mathieu,

thanks for reaching out; obviously I'd like to be part of this.

Having said that, I think we are actually quite well organized,
communicate a lot, meet regularly, and agree about more important things
than we disagree about.

I consider it a success that R Studio is actually working with us,
rather than developing its own spatial stack, and that it has traded old
concepts (package mapproj, fortifying polygons) for new ones brought in
by us (simple features, GDAL/GEOS/PROJ). I think R Studio and us both
see the benefits of having an R spatial stack that works well with
tidyverse, and we collaborate actively on this.

A communication platform that we (many of us) do use is
https://github.com/r-spatial/discuss (although I would agree we should
elaborate the goals of that repo, as well as the r-spatial.org website).

Also, a lot of R spatial projects have been funded by the RC and many
have been finished successfully; all of them have included a lot of
discussion while writing and received support from several members of
the R spatial community.

I'm not so sure where you see this lack of governance, and what you want
to build to fill that gap. I also don't see why this group would (start
with a) focus on tracking, while claiming to cover "spatial".

Many regards.

[basille]

Allow me to clarify a few points here:

• Edzer (and others) have provided a few interesting resources and pointers. Some of which were known, some not (like this discussion repository for instance, which I hope will be a good platform). Although that confirmed that the R spatial community is both very active and very successful, I think it also emphasize the scattered approach of the community (for instance, just looking at discussions here about the confusion on "rspatial.org" vs. "r-spatial.org" is fascinating — without any value judgment of the merits of both!).

• Most importantly, let me emphasize here that I expressed my opinion, as a movement expert, and with the bias associated to it. I can say that this opinion is at least shared by others who have expressed their interest in this. Maybe this is restricted to the movement world — and only applies to tracking data — and I would be happy to hear from other perspectives as well.

• It is true that the R Consortium has successfully funded several projects (e.g. sf, stars, geometa, mapview, our own sftrack package, etc.), but I understand from discussions with Hadley Wickham (for the R Consortium) that they are now limited by their own expertise to evaluate spatial proposals. We may have reached the point where individual proposals can be successful, and the RC would welcome guidance from the spatial data community.

• Finally, let me also clarify my remark about the lack of shared governance. It is true that a lot of development happens in the open, with fruitful discussions. But I think it is also true that it is very hard to understand where the spatial data community for R is going, and what is the perspective ahead of us. Again, this is certainly limited by my own perspective on movement.

I hope this leads to fruitful discussion here, as the whole point is to share opinions, highlight needs/gaps, and make it easier to reach some common goals.

[barryrowlingson]

The R Consortium (currently) supports two sorts of groups: Local User Groups and Working Groups (plus one-off things like R-Ladies and R-hub etc).

The Working Groups are ISC Working Groups:

"ISC working groups provide the mechanism through which the ISC can explore, fund, and manage large collaborative projects."

Where the ISC is the R Consortium's "Infrastructure Steering Committee."

Is this the proposed template for the Spatial Working Group? An R Consortium Spatial ISC Working Group?

Current ISC Working Group page lists five as "completed" which seem to be finished, and five "Active" which don't appear to be very active. For example the R/Pharma one links to an empty wiki page, and most of the others to github repositories with no commits for over a year.

I suppose the ISC Working Groups could be doing useful things behind the scenes or on other platforms, and reporting or advising the ISC (which is their job, see the governance docs for details) but I can't see that.

If R Consortium want to set up that sort of Working Group for Spatial then maybe some of us will contribute, but it is clear this is about the governance of Spatial for the R Consortium (ie what things do they spend their money on) and not for the R community as a whole.

We already have a thriving and productive R Special Interest Group which distributes itself across twitter, github, r-sig-geo and who knows where else. If the R Consortium wants to construct something more like that - something more akin to the Local User Groups - then I don't think that's necessary.

[edzer]

https://twitter.com/TimSalabim3/status/1244885177018376194

[rsbivand]

A couple of thoughts:

Before conceiving of a working group, the options for a mandate should be clarified. Starting a group is easier if there is some agreement on what "we" for some "we" would like it to achieve.

Before that again, one would need contrasting analyses of current status (probably plural analyses and conceptualisations of status). These differ greatly, so in preparing a mandate, one would have to look at which elements are more generally useful and inclusive, and which rather meet a specific set of needs (which will be easier to define).

The current informal SIG is/was a child of its time, and although ecological in structure (all welcome, no explicit structure), some of us are gate-keepers, trying to offer direction by encouragement while admitting that we can be wrong.

With respect to movement (a special kind of spatio-temporal data/analyses), the new task view looks promising. During 2020, the task view infrastructure is expected to become more open, so using task views more actively is one of the possibilities, though doing this wouldn't be a working group in the formal sense.

[tim-salabim]

Some relevant thoughts here
dcooley/sfheaders#64

[anitagraser]

Thank you for the invitation! I'm always interested in exchange concerning movement data analysis (even though my development background is in Python http://movingpandas.org). Therefore, I'm more interested in this more focused area than in the broader R spatial data working group.

[etiennebr]

I think what we need to hear is that the ISC finds it hard to evaluate spatial projects because they don't have the expertise. If they had this expertise, it would make it easier to fund larger spatial projects knowing that they will fit together to build a lasting spatial infrastructure for R.

We should give the Spatial Working Group a shot; I'm willing to join and contribute. In the best scenario, the SWG will take the time to clarify the vision for spatial in R and make it easier to align contributions beyond ISC grants. I don't see what could go wrong (I'm usually naive, though).

cc @angela-li

[rsbivand]

Another thought might be to structure a possible "working group" as an OSGeo community project in concert with interfacing ISC. In time, we might wish to incubate such a "project" as an OSGeo project, given that many OSGeo desktop projects try to integrate R, and we use most of the OSGeo geospatial libraries. Such a format would potentially be able to keep both sides in touch with ongoing changes, such as the current GDAL barnraising making most of R packages coordinate reference systems obsolete. @neteler - is this a really bad idea that should be dropped, or is some formalization of the position of R-spatial in relation to OSGeo worth at least considering? @basille - you and David know the rpostgis/PostGIS interface - might say your package benefit from some such structured interaction with the OSGeo framework?

[basille]

Thank you all for the very constructive feedback! I finally have time to get back to it, and follow up on all comments here. I like how this discussion is shaping, it will allow me to clarify what I suggested. I have started this on Twitter (you can read this thread if you're interested in more context), but let me try to do it here with more details, using comments contributed so far to fuel the discussion here.

[basille]

@barryrowlingson: Yes, what I was referring to is indeed an ISC working group, thanks Barry for the clarification (webpage here). One of the benefits would truly be for visibility and public recognition of the work and future directions of the R Spatial environment.

From discussions with Hadley Wickham, WGs are fairly loosely defined — constraints and organizational aspects would be on us, we would have to decide how we work, and what we want out of it. Some WG are certainly more active than others, I understand that there is a broad range out there (some being completely inactive). The R Consortium really only provides a roof and a badge. However, this could act as a way to provide feedback and guidance to the ISC on the efforts of the R spatial community so they can better evaluate proposals. So indeed, feedback going back to the ISC would be one of the primary goals of such a WG.

That actually allows me to clarify a second point: Such a WG would not have a goal of telling people what to do. R has benefited (and still benefits) a lot from individual/independent initiatives, and I would not want to lose that. Since you mentioned the R spatial SIG, that could simply be some sort of executive summary of what happens in this group at large, streamlining it publicly — and for the ISC.

[basille]

@etiennebr: Thank you for your support, Étienne! Indeed, the ISC expressed a need here, and acknowledged their own limits in terms of spatial expertise. The potential to fund larger spatial projects, or projects fitting into a larger perspective, is very relevant! For instance, in the feedback we received for the sftrack package, one thing that was very important was that it should have a broad impact. Not necessarily in terms of number of users affected, but most importantly in terms of scope, and how much this makes a change for the R infrastructure. This part is really hard to document, and makes it hard to really provide convincing metrics (fingers crossed for this round 🤞). I'm on the same Naive Team as you, so I want to be optimistic — but I'm also sure that things won't be that simple!

[basille]

@rsbivand: Thanks Roger, these are very good points! Let me state upfront that I recognize and value the role of gate-keepers. Changes for the sake of changing does not necessarily makes sense, and gate-keepers also provide a necessary "historical" perspective. I am actually glad that several people that I would consider as gate-keepers expressed their opinion here (or directly to me).

With the forthcoming changes in the CTV infrastructure, we can expect these to become more active and dynamic — maybe more open. The Tracking CTV is a great example of what we're trying to achieve broadly within the movement community: The Tracking CTV is designed to be an initiative that is community-led (via a template system to submit new packages on the GitHub repo), objective (explicit criteria for a package to be listed and kept in the list, or included into core), and dynamic (updated every 6 months).

But it could also multiply the CTVs of interest for the R spatial community. While this is certainly a good thing, it also makes it more complicated to navigate. There are already a number of main (Spatial, Spatio-temporal, Tracking) and related (Databases, Environmetrics, Hydrology) CTVs. We counted 58 packages in the tracking CTV alone, I'm not even willing to count the total number of spatial (and related) packages. Altogether, it really shows the diversity of the R spatial ecosystem, but also how hard it can be to navigate…

So the mandate of such a working group would really be to:

1. Provide a central repository of existing initiatives (e.g. the R-SIG-Geo mailing list, the CTVs, rspatial.org, r-spatial.org, etc.) as a guide for people to navigate the community and the projects;
2. Summarize/feed discussions about priorities and needs in development, potentially in subgroups (happy to push for a tracking/movement subgroup);
3. Regularly come up with a sort of executive summary of these discussions, of broad use but notably for the ISC.

As for building on OSGeo infrastructure, this would seem like a good idea to me! I have to admit I really don't know enough about OSGeo/OGC in terms of organization to comment on the feasibility of this, but that would certainly help in bridging gaps with other communities, and developing standards across platforms. Would you recommend an R community within OSGeo? Could you get me started on this, are there useful resources to start with?

[basille]

@anitagraser: Thanks Anita for stopping by! I think it is important to recognize that a lot happens outside of R (kudos to GeoPandas and MovingPandas)! I am personally more familiar with the database world through PostgreSQL/PostGIS notably, but there is a lot to learn from all other communities and tools, and I would personally be happy to have your feedback/perspective in a tracking subgroup!

[neteler]

@rsbivand, all: yes, some formalization of the position of R-spatial in relation to OSGeo is definitely worth considering! I support this idea.

[rsbivand]

Thanks @basille , constructive points. Re. OSGeo, I think this would need some thought, as the project model is usually for a single library or application. See: https://www.osgeo.org/ for a very broad-brush picture, and more detail in https://wiki.osgeo.org/wiki/Main_Page. I haven't found a page describing founding and administering a "project" yet, the term seems to be PSC, project steering committee. For GDAL, the current page is: https://gdal.org/community.html#governance-and-community-participation. An important mechanism is the RFC, which would match your point about discussing needs and proposals.

[neteler]

The point of contact in OSGeo is the "Incubation Committee" (https://www.osgeo.org/about/committees/incubation/) - I agree that it would need higher visibility in the menu of the OSGeo web site. Details are described here: https://wiki.osgeo.org/wiki/Incubation_Committee with an overview here: https://www.osgeo.org/about/committees/incubation/incubation-process/

[edzer]

Thanks, Markus! We could propose an "R Spatial" OSGEO project consisting of a set of R packages, for instance those on this GH organisation + sp/rgdal/rgeos, while being open for including more packages in that set (and include raster, if @rhijmans wants to),

Most of the questions on the incubation application questionnaire seem rather straightforward, but 18. If you do not intend to host any portion of this project using the OSGeo infrastructure, why should you be considered a member project of the OSGeo Foundation? is something to think about.

A number of points that I can come up with to answer this question:

• to be clear which packages are in the scope of this project
• to be clear which authors are (or have been) involved, and which licenses are used for what
• to formalize a PSC which makes decision making more transparent
• to handle releases and their effect on reverse dependencies more transparently and efficiently
• to improve communication with other OSGEO projects, in particular upstream OSGEO projects used by R Spatial, and downstream OSGEO projects that integrate R
• to offer an R Spatial point of contact for the R community

[neteler]

(I don't think that question 18 is that important... a number of OSGeo projects use their own infrastructure).

[barryrowlingson]

I don't know why the R Consortium is asking us mere mortals about spatial data when they have the Esri behemoth as partners. I imagine they could spare a few people for a Consortium steering committee.

Anyone else catch this badly-written Esri Lack-of-info-mercial from the Consortium:

https://www.r-consortium.org/blog/2020/04/07/esri-empowers-informed-decision-making-covid-19

Oh its a badly-formatted cut n paste job from an Esri page. Lazy.

[basille]

@barryrowlingson: Obviously, you don't hold the R Consortium close to your heart, and I very much respect that. However, I wonder how this idea of a R Spatial Working Group has anything to do with ESRI… The R Consortium explicitly acknowledged a lack of expertise to evaluate spatial proposals — certainly this expertise cannot come from ESRI but only from the R community itself. The fact that the R Consortium allows this kind of advertising from ESRI is also irrelevant. Let me clarify one point here: nobody is asking anyone in the community to give up their independence to the R Consortium. Clarifying what we're doing, and where we're heading, is the goal.

@rsbivand, @neteler, @edzer: I'm looking into this right now.

[basille]

@edzer provided very good points to get started. I think we should start drafting our thoughts, and see with OSGeo what's missing… @neteler, is there a place that we could use for this, maybe on the OSGeo wiki? This thread does not seem appropriate for such a draft — maybe a GH wiki here? (I don't have much experience with those)

A few things that seem of utmost importance to me, before we get started:

• It should be about the community behind the R spatial ecosystem. But that is very vague… How do we decide who gets in and who does not? And who is "we" here?
• Corollary question: Surely, there should be some criteria to qualify R packages… But what would those be exactly? Provides spatial data structures, or connectors/converters to spatial data structures?
  Or shall we leave all this open, and simply welcome every volunteer who feels they "belong" to this community? (it would also be very good to have outside observers too)

Also, it is not entirely clear to me whether "community projects" and "OSGeo projects" (as presented on the OSGeo Incubation Committee page here) are just two designations for the same thing, or are they two different things? @neteler?

[etiennebr]

I've found that the easiest way to review and comment on a document is to use a PR. It could be here or a personal repo.

[neteler]

Also, it is not entirely clear to me whether "community projects" and "OSGeo projects" (as presented on the OSGeo Incubation Committee page here) are just two designations for the same thing, or are they two different things? @neteler?

They are not the same: to reach a "community project" is fairly easy (this does not include access to OSGeo funding or infrastructure support etc.; see also https://wiki.osgeo.org/wiki/OSGeo_Community_Projects) while obtaining the status of "OSGeo project" requires an incubation phase with code vetting, license check, community "health" check etc and the decision by the board to graduate the candidate project.

[rsbivand]

I feel that this is relevant to this discussion issue: https://lists.osgeo.org/pipermail/grass-dev/2020-June/094453.html, pointing to https://wiki.osgeo.org/wiki/SAC_Shared_Building_Services. How might we map this out? Now, CRAN binary builds should be static linked to reduce/eliminate external dependencies and reduce/eliminate license issues when distributing binary packages. Who might collaborate to explore this?

[rsbivand]

Link to R-sig-geo posting and thread: https://stat.ethz.ch/pipermail/r-sig-geo/2020-June/028214.html. Text from us now in place in the SAC wiki. Need to establish a handshake-bridge so that they can contact us and we can contact them.

[basille]

Thank you @rsbivand for the follow-up — it is indeed relevant! Unfortunately, I haven't been able to work on a proposal for R-Spatial as a project associated to OSGeo, but I hope to be able to do so over the summer (my current situation does not allow for any visibility at the moment).

I do appreciate that you and others keep adding to this discussion. This is a very interesting conversation with many positive aspects involved.

[eblondel]

Dear all, I just discovered this ticket, and I support the idea of having a working group related to R spatial. I also had an exchange with Hadley Wickham, after my 2d proposal to R consortium was not accepted last year (the 1st one, accepted 2 years ago, was focused on geometa and in a less extent ows4R ). In particular, it was said that The ISC is supportive of further advancements in R for spatial data, however as it is a big domain the ISC would like to see a coordinated strategy from the spatial community on which aspects should be prioritised. Forming a ISC Working Group could be an option to facilitate that.
In any case i'd be happy to contribute to discussions and proposals that could help federating on r-spatial matters.
Best
Emmanuel

[Robinlovelace]

Follow-up after a panel discussion (available on YouTube here). Looking at the link above from @neteler it seems we should follow the guidance here to become an OSGeo Community Project: https://wiki.osgeo.org/wiki/OSGeo_Community_Projects

Specifically it says

If you would like your software project to listed on the website, or apply to become an OSGeo community project, please send a description of your project to the OSGeo Incubation Committee Mailing List. We look forward to helping you!

Looking at the archives it seems no request has been made for an R-Spatial Community project: https://lists.osgeo.org/pipermail/incubator/

@edzer raised a potential issue with 'question 18' that does not seem to be an issue. One different but related question is how R is packaged in the OSGeo live distro - I tried back in 2016 to get RStudio included as that's the most accessible and user friendly IDE for R-spatial work - OSGeo/OSGeoLive#109 - something we could pick-up on (see discussion on their mailing list here).

None of these things prevent us moving forward. @rsbivand asked who would be up for helping to move this forward. I'm up for it. Should we draft an email? Up for giving it a 1st bash unless anyone else involved in the community and involved in the above discussion, e.g. Edzer/Roger, would like to. I think as a starting point 'the repos in the r-spatial github org that are on CRAN' is fine and we can cross the bridge of how to add other packages such as raster/terra when we get to it and ask it as an open question (I'm in favor of including those assuming package authors agree).

All this will help grow the r-spatial community and build/strengthen bridges to other OSGeo projects, not least GDAL, PROJ and QGIS.

[Robinlovelace]

Update, I've split-out the issue of OSGeo membership in #44. This thread has covered a lot of ground but I think should re-focus on the original topic, assuming it's still an open discussion.

[joseph-rickert]

Hello All,
I encourage you to apply to become a working group under the R Consortium. This would provide you with a company neutral platform for organizing your open source work, enable you to receive and administer grant money, and form partnerships with other organizations. Note that over the past four years, the R Consortium has awarded more than $100K in spatial statistics related grants.

[rsbivand]

@joseph-rickert Please see #44 , the OSGeo Community Project application is under way, and that will be the same as the working group wrt. RC. Is there and application process to become a working group, should we simply inform RC of the progress of the OSGeo application?

[joseph-rickert]

Hi Roger, R Consortium working groups are intended for inter-company collaboration. The R Consortium essentially vouches to supervise the wg as an open, company neutral collaboration and allocates some basic resources like a GitHub repo to facilitate getting the work done. The R Validation Hub <https://www.pharmar.org/> and the Submissions <https://github.com/RConsortium/submissions-wg> working group are both good examples. Working groups need to be approved by the ISC <https://www.r-consortium.org/about/governance>. Not all working groups are funded. Some just need organizational help, others do need funding. To apply for working group status without ISC funding write to Hadley Wickham, the ISC Chair, and he will bring it before the committee. Assuming the ISC believes that the group would significantly contribute to the infrastructure supporting the R Community and approves the group, this would be an easy way to get started, and you can always seek funding later. To apply to become a working group with ISC funding, the process is to submit a proposal to the ISC when the call for proposals opens and ask to be a working group in addition to asking for a grant. I do not know anything about how OSGeo projects are run, but I assume that you could operate as an ISC working group and receive grants from both the ISC and OSGeo. Best regards, Joe Joseph B. Rickert RStudio R Community Ambassador R Consortium Director Cell: 408.489.0566

…

On Wed, Mar 3, 2021 at 9:38 PM Roger Bivand ***@***.***> wrote: @joseph-rickert <https://github.com/joseph-rickert> Please see #44 <#44> , the OSGeo Community Project application is under way, and that will be the same as the working group wrt. RC. Is there and application process to become a working group, should we simply inform RC of the progress of the OSGeo application? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#37 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN22VTZDW74LCINLRDA2W3TB4MFLANCNFSM4LWDZDAA> .

[rsbivand]

@joseph-rickert Thanks! Heads up for others in the discussion, @Robinlovelace - did the proposal get sent to OSGeo, and do we have a timeline that would match Joe's sugggestions https://www.r-consortium.org/projects/isc-working-groups ? If 2021 matches 2020, the first cycle for applications should open soon, and close early April (or wait until the second cycle in September). I think we'd go for unfunded for the present, because planning face-to-face interaction is too uncertain.

This maybe also says something to the ROpenSci standards book project https://github.com/ropenscilabs/statistical-software-review-book/blob/main/standards/spatial.Rmd @Nowosad and ongoing thinking at https://opengeohub.org/ (does Tom Hengl have a handle here?). OpenGeoHub would be a good partner for an R Consortium Working Group, as would Mundialis and others.

[Nowosad]

@thengl