Skip to content

Review/daniel farrell/13709 #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
edwarnicke wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Review/daniel farrell/13709 #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
dc56238
Ignore URLs in line length linting, docs build dir
dfarrell07 Nov 28, 2018
ecb2ce4
Add LNF initiatives suggestions, CII/Sec/CI/CD
dfarrell07 Nov 28, 2018
dbf5d9f
Add Infra Update as topic
edwarnicke Jan 16, 2019
a861f41
Trigger DCO CI
edwarnicke Jan 16, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions .coafile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
[all]
ignore = .tox/**,
.git/**,
ignore = .git/**,
.gitignore,
.gitreview,
.tox/**,
node_modules/**,
**.sw?,
**.orig
docs/_build/**,
**.orig,
**.sw?

[all.reStructuredText]
bears = RSTcheckBear
Expand All @@ -15,6 +16,8 @@ files = **.rst
bears = LineLengthBear
files = **.rst, **.txt
max_line_length = 79
ignore_length_regex = http://,
https://

[all.spacing]
bears = SpaceConsistencyBear
Expand Down
74 changes: 74 additions & 0 deletions docs/initiatives/suggestions.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
===============================
Suggestions for LFN Initiatives
===============================

Update Infra
------------

LFN Projects are using a Gerrit+Jenkins+Nexus infra that is both antequated and
expensive to maintain. More modern open source projects tend to use Github based
infra like:

* Github - Source Code Management
* CircleCI or TravisCI or other cloud based CI - CI
* Various artifact repo as a service services
* Github Issues - Bug tracking
* Github + Hugo + Netlify - Websites/doc sites
Copy link

@morganrOL morganrOL Jan 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as mentioned in the discussion thread (#1)
on our side we moved to gitlab because most of the features mentioned though github + .. are built-in.
we use the following features

  • Source code management (very similar to github)
  • CI/CD
  • bug tracker
  • website
  • artifact management

Moreover gitlab is open source, which provides flexibility.

Copy link
Collaborator Author

@edwarnicke edwarnicke Jan 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@morganrOL Happy to throw gitlab into the consideration as well. The thing I care most about personally is ecosystem of tools. Github is the leader there, but gitlab is a close second (and has lots of other cool features as you mention).


This subgroup will explore cross project how to POC, and perhaps migrate LFN
projects to such more modern infra.

It is important to be clear: no community should change if it does not see benefit.
Copy link

@bdfreeman1421 bdfreeman1421 Jan 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to specifically evaluate if moving to a CI as a Service approach would be lower cost and meet all the security and access requirements. For those consumers of opensource would github be easier to work with - for example companies already usin git would moving to github from gerrit allow preservation of commit history etc which is lost in the git to gerrit pushes that occur today

Task should include report on:

  1. Cost benefit to projects moving to CI as a Service
  2. Required tool changes to realize that cost savings
  3. New Flows documented for both PTLs and contributers with the new tooling
    a. changes from git commit -as --amend and git review
    b. changes to Jira plugins
    c. any changes to pom.xml ?
    d. conversion of jjb to another tool
  4. How will repo creation be managed so that project requirements for accountability are maintained
  5. How will access control be maintained so that linux foundation ID's with Contributer Agreement relationship is maintained or replaced with a different strategy ?

Copy link
Collaborator Author

@edwarnicke edwarnicke Jan 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bdfreeman1421 Excellent questions! I have some partial answers:

for example companies already usin git would moving to github from gerrit allow preservation of commit history etc which is lost in the git to gerrit pushes that occur today

Github has flexiblity here. You can pick how PRs get merged either PR by PR, or at the repo level. Options are:

  1. Merge
  2. Squash and merge
  3. Rebase and merge

Squash and merge has the 'collapse the history from the requested change into a single commit' behavior. I personally prefer squash and merge, but communities can decide their own preferences.

Task should include report on:

  1. Cost benefit to projects moving to CI as a Service
  2. Required tool changes to realize that cost savings
  3. New Flows documented for both PTLs and contributers with the new tooling
    a. changes from git commit -as --amend and git review
    Yep... the equivalent tool for github would be hub
    b. changes to Jira plugins
    Atlantsian appears to be actively improving their Jira github integration
    c. any changes to pom.xml ?

Changes to pom.xml would depend on whether a community chose to switch to a cloud based maven repo provider, there are a bunch: sonatype provides free maven hosting for open source projects, packagecloud.io provides free hosting for open source projects (though at your scale, we may have to pay them a bit),deps.co offers maven hosting as a service. Not sure which if any of these would meet your communities needs... but there are a lot of good choices.

d. conversion of jjb to another tool
Yep. Thought I must say, having worked extensively with jjb and with circleci and travisci... circleci and travis are both simpler and more powerful... and the jobs are defined right in the project repos, so no more having to coordinate patches with a seperate jjb repo makes things go much much faster and smoother.

  1. How will repo creation be managed so that project requirements for accountability are maintained

One way to approach the problem would be to simply give the TSC members (or some subset chosen by the projects TSC) Admin rights to their github orgs. This way, when a project is approved by the TSC, a TSC member can create the repo right there on the spot.

  1. How will access control be maintained so that linux foundation ID's with Contributer Agreement relationship is maintained or replaced with a different strategy ?

There are tools to require DCO for github (we are using one in this repo), and a bunch of available tools for enforcing CLA (I've had to go through them at times when contributing to a project on github). So eminently solvable :) But an important point to bring up.


CII Badging
-----------

LNF Projects should work together to meet CII Badging requirements.

CII Badging covers many aspects of project quality along an increasing
difficulty scale from Passing to Silver to Gold. At the high end of the scale,
the requirements are exceptionally difficult (binary-reproducible builds, 2FA
for all commits).

The CII Badging requirements are here:

https://github.com/coreinfrastructure/best-practices-badge/blob/master/README.md

ONAP has extensive CII Badging docs here:

https://wiki.onap.org/display/DW/CII+Badging+Program

Cross-Project Security
----------------------

Track and quash security vulnerabilities in LFN leveraging cross-project
advantages.

Cross-Project CI/CD
-------------------

Continue improving and extending LFN Cross-Project Continuous Integration
Testing (CI) and Continuous Delivery (CD) pipelines.

Examples:

* Provide continuous, automated builds in common package formats for
consumption by other projects.
* Provide common configuration management tooling to facilitate installation
and configuration by other projects.
* Provide pre-built/installed/configured containers to facilitate deployments
by other projects.
* Run tests that consume LFN Project's CD pipelines and tooling, potentially
integrating multiple LFN Projects, and validate deployment scenarios.

Identify Quality Tooling
------------------------

Leverage the shared knowledge and experience of LFN Projects to identify good
solutions to common tooling problems.
Copy link

@morganrOL morganrOL Jan 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are lots of interesting linters available, vrpg created a docker which integrates lots of upstream linters
https://hub.docker.com/r/vpgrp/linter
(markdown, ansible, python, docker, yaml, prometheus, go, riby, shell,...).
each community may obviously have its own rules and customization.


As always, LFN Projects are completely in control of what tooling they choose
to use.

Examples:
* Better options than Nexus IQ for scanning