Skip to content

Commit

Permalink
Merge pull request #17 from NHSDigital/SH_new_front_page
Browse files Browse the repository at this point in the history
Merging after approval
  • Loading branch information
SamHollings authored Aug 15, 2022
2 parents 7131fd0 + 75ef82a commit 953e9dc
Show file tree
Hide file tree
Showing 15 changed files with 75 additions and 26 deletions.
77 changes: 51 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,18 @@
***Warning - this repository is a snapshot of a repository internal to NHS Digital.
This means that some links may not work for external readers.***
[what is RAP]: what-is-RAP.md
[Publishing Code]: development-approach/08_how-to-publish-your-code-in-the-open.md
[Example Repos]: https://github.com/NHSDigital/data-analytics-services#rap-repositories
[Training your team]: /training
[Git]: /development-approach
[Python]: /python
[R]: /R
[Testing]: /development-approach
[Coding Best Practice]: /coding-best-practice
[RAP Community of Practice Teams page]: https://teams.microsoft.com/l/channel/19%3aEnoJ_c3NIwcWaLoqHyrbmiui8SI-8lZ1B2SvxNuGbyU1%40thread.tacv2/General?groupId=1c1528d7-030e-48eb-92cf-dc0f6a618ea0&tenantId=50f6071f-bbfe-401a-8803-673748e629e2

| | | | | |
|---|---|---|---|---|
| [<img alt="What is RAP?" img src="images/What is RAP icon.png">][what is RAP] | [<img alt="Publishing Code" img src="images/publishing code icon.png">][Publishing Code] | [<img alt="Python" img src="images/Python icon.png">][python] | [<img alt="R" img src="images/R icon.png">][R] | [<img alt="Coding Best Practice" img src="images/coding best practice icon.png">][Coding Best Practice] |
| [<img alt="Example Repos" img src="images/Example Repos icon.png">][Example Repos] |[<img alt="Training your team" img src="images/training your team icon.png">][Training your team] | [<img alt="Git" img src="images/Git icon.png">][git] | [<img alt="Testing" img src="images/Testing icon.png">][Testing] | [<img alt="NHSD Analytics Function" img src="images/NHSD Analytics Function repo.png">](https://github.com/NHSDigital/data-analytics-services) |

***Repository owner: [NHS Digital Analytical Services](https://github.com/NHSDigital/data-analytics-services)***

Expand Down Expand Up @@ -30,22 +43,28 @@ Luckily, at NHS Digital we have strong senior support for RAP and several teams
## Support
If your team is embarking upon a RAP journey, you should look at our [what is RAP](what-is-RAP.md) page and think about which [levels of RAP](what-is-RAP.md#levels-of-rap) that you want to target.

From there, we recommend reaching out for some in-person support. The [RAP Champions Network](https://analysisfunction.civilservice.gov.uk/support/reproducible-analytical-pipelines/reproducible-analytical-pipeline-rap-champions) can offer support through:
* helping you understand why and how they implemented RAP in your department and why it worked for them
* sharing code via platforms like GitHub so that others can learn from and adapt what has been done
* shadowing & mentoring opportunities
* peer review
From there, we recommend reaching out for some in-person support.

If you work at NHS Digital, get in touch! We you can offer some in-person support in several forms:
**If you work at NHS Digital**, the **Data Science RAP squad are your main RAP Champions: get in touch!** We you can offer some in-person support in several forms:
* Reviewing your RAP work and assessing your progress against the [levels of RAP](what-is-RAP.md#levels-of-rap)
* Peer review of code
* Workshops for a specific RAP capability
* Consultancy style engagement where we plan a migration strategy
* Pair coding
* Shadowing another team

There is also a growing community of RAP experts outside the Data Science RAP squad - you can find them on our [RAP Community of Practice Teams Page] (NHS Digital only)

**If you're not in NHS Digital**, the [RAP Champions Network](https://analysisfunction.civilservice.gov.uk/support/reproducible-analytical-pipelines/reproducible-analytical-pipeline-rap-champions) can offer support through:
* helping you understand why and how they implemented RAP in your department and why it worked for them
* sharing code via platforms like GitHub so that others can learn from and adapt what has been done
* shadowing & mentoring opportunities
* peer review

They also have more guidance on how to get started, and get help [here](https://dataingovernment.blog.gov.uk/2022/08/08/mentoring-a-successful-rap-project/).

There are several forums where you can introduce yourself, ask for help, or discuss different approaches:
* NHS Digital RAP [MS Teams page](https://teams.microsoft.com/l/channel/19%3aEnoJ_c3NIwcWaLoqHyrbmiui8SI-8lZ1B2SvxNuGbyU1%40thread.tacv2/General?groupId=1c1528d7-030e-48eb-92cf-dc0f6a618ea0&tenantId=50f6071f-bbfe-401a-8803-673748e629e2) (internal to NHS Digital).
* NHS Digital RAP [MS Teams page][RAP Community of Practice Teams page] (internal to NHS Digital).
* [RAP Slack channel](https://nhsrcommunity.slack.com/archives/C03N1GXHEH0) from the [NHS-R](https://nhsrcommunity.com/) community.
* [RAP collaboration Slack channel](https://govdatascience.slack.com/archives/C6H22U3H9) on the cross-government data science Slack Workspace (open to all civil servants, including NHS staff).

Expand All @@ -67,23 +86,27 @@ These resources are demand-driven so if you think of something worth including,
These resources are not intended to be prescriptive. There are many ways to accomplish a task and teams have valid reasons for choosing other approaches. We hope that these resources are helpful for teams wanting to adopt good practices that they have heard about but don't know where to start.

## External links
Here is a good [high-level overview of RAP](https://dataingovernment.blog.gov.uk/2017/03/27/reproducible-analytical-pipeline/). It explains the context of RAP in government statistics and gives some history to the work.

The GSS have produced a list of the [benefits that come from RAP](https://gss.civilservice.gov.uk/reproducible-analytical-pipelines/benefits-to-government-from-reproducible-analytical-pipelines/).

The [ONS best practice team have a useful website](https://best-practice-and-impact.github.io/qa-of-code-guidance/intro.html) covering many of the same topics we cover here. Their [best-practice checklist](https://best-practice-and-impact.github.io/qa-of-code-guidance/checklist_higher.html) is particularly useful.

The Statistics Authority has published this excellent report on [overcoming barriers to RAP adoption](https://osr.statisticsauthority.gov.uk/publication/reproducible-analytical-pipelines-overcoming-barriers-to-adoption/).

The [AQUA book of analytical standards](https://www.gov.uk/government/publications/the-aqua-book-guidance-on-producing-quality-analysis-for-government).

The Turing Institute has published [The Turing Way handbook to reproducible, ethical and collaborative data science](https://the-turing-way.netlify.app/welcome.html)

There are several slack channels that discuss RAP and related topics: the [govdatascience.slack.com RAP channel](https://govdatascience.slack.com/archives/C6H22U3H9), the [NHS-R community](nhsrcommunity.slack.com), and the [NHS-pycom community](nhs-pycom.slack.com)

Alston, J. M., and Rick, J. A.. 2020. A Beginner's Guide to Conducting Reproducible Research. Bull Ecol Soc Am 102(2):e01801. https://doi.org/10.1002/bes2.1801

The GSS training portal offers an [Introduction to RAP course](https://gss.civilservice.gov.uk/training/introduction-to-reproducible-analytical-pipelines-rap/)
### Strategic
- The Government Analysis Function has published their [RAP strategy](https://analysisfunction.civilservice.gov.uk/policy-store/reproducible-analytical-pipelines-strategy/). It's very good for understanding what role we all have to play in RAP.
- The [Goldacre Review](https://www.gov.uk/government/publications/better-broader-safer-using-health-data-for-research-and-analysis) mentions RAP extensively as a key tool in improving the way we use patient data.
- [Here](https://www.bennett.ox.ac.uk/blog/2022/07/bennett-insights-an-overview-of-uk-data-policy-developments/) is a helpful summary of the above documents and others which helps understand RAP and why it is important in the context of UK data policy developments.

### Guidance and Standards
- The [AQUA book of analytical standards](https://www.gov.uk/government/publications/the-aqua-book-guidance-on-producing-quality-analysis-for-government).
- The [ONS best practice team have a useful website - often called the Quack book](https://best-practice-and-impact.github.io/qa-of-code-guidance/intro.html) covering many of the same topics we cover here. Their [best-practice checklist](https://best-practice-and-impact.github.io/qa-of-code-guidance/checklist_higher.html) is particularly useful.
- The Turing Institute has published [The Turing Way handbook to reproducible, ethical and collaborative data science](https://the-turing-way.netlify.app/welcome.html)

### RAP Examples, Challenges, Benefits
- Here is a good [high-level overview of RAP](https://dataingovernment.blog.gov.uk/2017/03/27/reproducible-analytical-pipeline/). It explains the context of RAP in government statistics and gives some history to the work.
- The GSS have produced a list of the [benefits that come from RAP](https://gss.civilservice.gov.uk/reproducible-analytical-pipelines/benefits-to-government-from-reproducible-analytical-pipelines/).
- The Statistics Authority has published this excellent report on [overcoming barriers to RAP adoption](https://osr.statisticsauthority.gov.uk/publication/reproducible-analytical-pipelines-overcoming-barriers-to-adoption/).
- Alston, J. M., and Rick, J. A.. 2020. A Beginner's Guide to Conducting Reproducible Research. Bull Ecol Soc Am 102(2):e01801. https://doi.org/10.1002/bes2.1801
- The GSS training portal offers an [Introduction to RAP course](https://gss.civilservice.gov.uk/training/introduction-to-reproducible-analytical-pipelines-rap/)
- The Government Analysis Function also did some [research on the rollout of RAP across different deparments](https://best-practice-and-impact.github.io/CARS-3/index.html).

### Social spaces
- There are several slack channels that discuss RAP and related topics: the [govdatascience.slack.com RAP channel](https://govdatascience.slack.com/archives/C6H22U3H9), the [NHS-R community](nhsrcommunity.slack.com), and the [NHS-pycom community](nhs-pycom.slack.com)
- We have an [MS Teams page](https://teams.microsoft.com/l/channel/19%3aEnoJ_c3NIwcWaLoqHyrbmiui8SI-8lZ1B2SvxNuGbyU1%40thread.tacv2/General?groupId=1c1528d7-030e-48eb-92cf-dc0f6a618ea0&tenantId=50f6071f-bbfe-401a-8803-673748e629e2) (internal to NHS Digital)

## Misc
We have taken inspiration from the [NHS Digital software engineering COP](https://github.com/NHSDigital/software-engineering-quality-framework/blob/master/insights/review.md). It has tons of great material so I encourage you to read and reflect on these working practices.
Expand All @@ -93,3 +116,5 @@ RAP Community of Practice codebase is released under the MIT License.

The documentation is © Crown copyright and available under the terms of the [Open Government 3.0](https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/) licence.

***Warning - this repository is a snapshot of a repository internal to NHS Digital.
This means that some links may not work for external readers.***
12 changes: 12 additions & 0 deletions coding-best-practice/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Coding Best Practice
This page is still under construction, however you can find guidance on this at these pages:
- General Code Development
- [Unit testing, back testing, tidy data and notebook vs IDE development](/development-approach)
- Python
- [Our general Python guidance incl. guidance on why to use functions!](/python)
- [Pyspark style guide](/pyspark)
- [Very comprehensive Python guidance here on almost every aspect of python (external link)](https://docs.python-guide.org/)
- [google python style guide, which our Pyspark style guide is heavily based on](https://github.com/google/styleguide/blob/gh-pages/pyguide.md)

- R
- [R guidance - in progress](/R)
Binary file added images/Example Repos icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/Git icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/Levels of RAP icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/NHSD Analytics Function repo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/Python icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/R icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/Testing icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/What is RAP icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/Why RAP icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/coding best practice icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/publishing code icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/training your team icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
12 changes: 12 additions & 0 deletions training/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[RAP Champions]: ../README.md#support

# Training your team
This is just a stub page, but soon we intend to populate this with guidance on how to approach training your team in RAP.

Some initial pointers to get you started:
- **Get your team learning the basics of an open source language and Git**: confidence using version control, and a programming language of choice (at NHS Digital, this is usually Python), will make it easier for your team to engage with RAP training.
- **Make time for RAP**: people need time to learn and use what they've learned - if they don't get time set aside specfically for RAP they won't develop these skills, and will potentially fall behind and lose motivation.
- **Pragmatism is ok!:** You don't leap up to the level of full pyspark pipelines in one go. There is nothing wrong with starting very basic: wraping your existing SQL code in a bit of Python to orchestrate it, get that working, include some very simple tests to prove it works and then add it to Git and publish it. This is a great way to start and provides a solid foundation on which to build and learn more.
- **Copy other people:** Look at your colleagues code, copy chunks of it, play around with it, try and break it - encourage others do the same. Re-use is at the heart of RAP and it's also a great way to teach and learn.
- **Talk to the RAP champions:** The RAP champions (both your RAP trained colleagues and the Data Science RAP squad) have probably trained a number of people - ask them for advice. You can find out how to get in touch [here][RAP Champions].
- **The thin-slice approach is built with training in mind:** [See our pages on the the "thin slice" approach of doing RAP](/rollout-approach/thin-slice-strategy.md) - by doing a simple part of the publication from end to end, you create something people can learn from, and then with help expand. This can provide the structure around which you can train people.

0 comments on commit 953e9dc

Please sign in to comment.