Add vertex attributes (updated) #93

clhunsen · 2018-02-16T09:38:36Z

This is the updated version of PR #67, bringing some important improvements (diff:
https://github.com/flx5/codeface-extraction-r/compare/vertex-attributes...clhunsen:vertex-attributes). Basically, this tackles #34.

Thanks to @flx5 for this high effort! 😃

The open issues related to vertex attributes, all originating from the discussion in PR #67, are collected in issue #92.

Add methods required to add vertex attributes to the graph. Add method to calculate commit count as vertex attribute Signed-off-by: Felix Prasse <[email protected]>

Signed-off-by: Felix Prasse <[email protected]>

- Rename aggregation to aggregation.level and all to project - Rename add.vertex.attributes.* methods to add.vertex.attribute.* - Rename compute.vertex.attribute.with.n2r to add.vertex.attribute - Rename compute.vertex.attribute to split.and.add.vertex.attribute - Move networks.to.ranges to util-split and rename it to split.data.by.networks - Remove explicit return statement where possible - Add missing documentation Signed-off-by: Felix Prasse <[email protected]>

…nd issues Signed-off-by: Felix Prasse <[email protected]>

Signed-off-by: Felix Prasse <[email protected]>

Add test for add.vertex.attribute Add test for split.and.add.vertex.attribute Add test for add.vertex.attribute.commit.count Add test for add.vertex.attribute.author.email Add test for add.vertex.attribute.artifact.count Add test for add.vertex.attribute.first.activity Signed-off-by: Felix Prasse <[email protected]>

The tests were referencing to a wrong project data variable Signed-off-by: Felix Prasse <[email protected]>

Signed-off-by: Felix Prasse <[email protected]>

Add test for add.vertex.attribute.author.role.simple Add test for add.vertex.attribute.active.ranges Signed-off-by: Felix Prasse <[email protected]>

Signed-off-by: Felix Prasse <[email protected]>

Add artifact attributes Add commit count based on committer Signed-off-by: Felix Prasse <[email protected]>

Signed-off-by: Felix Prasse <[email protected]>

In this patch, we mainly streamline the tests on network attributes, by applying proper line breaks, introducing global constants, and fixing the accidental introduction of factors for the developer roles. This tackles some comments in PR #67. Signed-off-by: Claus Hunsen <[email protected]>

With respect to PR #67, logging statements, line breaks, and capitalization in sqldf-statements are updated. Signed-off-by: Claus Hunsen <[email protected]>

With respect to PR #67, indentation, line breaks, coding flaws, and anything else is tackled. Adds a FIXME item regarding the first activity of an author, which will be tackled in an upcoming commit. Signed-off-by: Claus Hunsen <[email protected]>

The section dividers always end at column 77. ;) Signed-off-by: Claus Hunsen <[email protected]>

Signed-off-by: Claus Hunsen <[email protected]>

@bockthom

To get the very first and the very last activity across all data sources, i.e., the earliest/latest timestamp of all data sources, for a ProjectData object, we add a parameter 'outermost' to 'ProjectData$get.data.timestamps' that makes this possible. Props to @bockthom for his support. Signed-off-by: Claus Hunsen <[email protected]> Signed-off-by: Thomas Bock <[email protected]>

@bockthom

Formerly, we provided three different aggregation levels in the function 'split.data.by.networks': namely, "range", "cumulative", and "project". We now extend and adapt this to the levels "range", "cumulative", "all.ranges", "project.cumulative", "project.all.ranges", and "complete". The various aggregation levels work as follows: - range: The project data will be split exactly to the time ranges specified by the networks' names. - cumulative: The project data will be split exactly to the time ranges specified by the networks' names, but in a cumulative manner. - all.ranges"}: The project data will be split exactly to the time range specified by the start of the first network and end of the last network. All data instances will contain the same data. - project.cumulative: The same splitting as for \code{"cumulative"}, but all data will start at the beginning of the project data and *not* at the beginning of the first network. - project.all.ranges: The same splitting as for \code{"all.ranges"}, but all data will start at the beginning of the project data and *not* at the beginning of the first network. All data instances will contain the same data. - complete: The same splitting as for \code{"all.ranges"}, but all data will start at the beginning of the project data and end at the end of the project data. All data instances will contain the same data. In this patch, we properly extend all functions and tests. Props to @bockthom for his support in parts of this patch. Signed-off-by: Claus Hunsen <[email protected]> Signed-off-by: Thomas Bock <[email protected]>

The function 'add.vertex.attribute.author.role' now adds author- classification attributes based on classification results. The function 'add.vertex.attribute.author.role.function' is now the one using a classification function as a parameter. For this to work, we add the range identifier as a parameter to the anonymous calculation functions for adding the vertex attributes. Signed-off-by: Claus Hunsen <[email protected]> Signed-off-by: Thomas Bock <[email protected]>

Signed-off-by: Claus Hunsen <[email protected]> Signed-off-by: Thomas Bock <[email protected]>

For better modularity, the function 'get.commit.data' is moved to the data module, right where it obviously belongs. Signed-off-by: Claus Hunsen <[email protected]>

@flx5

This code originates from @flx5's PR on vertex attributes (#67). Signed-off-by: Claus Hunsen <[email protected]>

bockthom

After finally reviewing this pull request, let me conclude:

This pull request looks pretty good, but there is still room for improvement. Basically, there are two different types of issues:

Wrong or insufficient documentation (needs to be fixed, some of my comments may be too meticulous -- @clhunsen it is up to you whether you address all of them or just the important ones)
General consistency problems regarding copyright headers (discussion needed, see individual comments)

Please don't be angry about me for spotting all the small stuff...

bockthom · 2018-02-18T14:32:16Z

util-data.R

+#' @param split A list of numerics, indicating numbers of weeks into which the selected data
+#'              is to be split
+#'
+#' @return A data.frame indicating the slected \code{columns}, split into the given numbers


Typo in "slected"

bockthom · 2018-02-18T14:43:25Z

tests/test-split.R

 ## [email protected]
+## (c) Felix Prasse, 2017
+## [email protected]
+## (c) Thomas Bock, 2018


I'm not sure why you have added my name in all of the copyright headers in those files. Maybe it is due to the Signed-Off-Tags. Currently, in the released version, my name is not part of any copyright header (except for the network-metrics script). If you would like to make this consistent, we would have to search for all Signed-Off-Tags containing my name and adapt the copyright headers of the corresponding files accordingly...

Long story short, I don't like inconsistencies. Either we remove my name from all copyright headers in this pull request or I will search for all Signed-Off-Tags containing my name to make all copyright headers consistent (not any more in this pull request, but possibly in one of the next ones...)

In addition, this affects also commit 4810ac5, in which the function get.commit.data is moved from util-core-peripheral.R to util-data.R: The function get.commit.data originally was implemented by Ferdinand. Ferdinand is part of the copyright header of util-core-peripheral.R (which still contains functions implemented by him), but Ferdinand is not part of the copyright header of util-data.R. As the moved function was implemented by him, he should also get part of the copyright header there...

My suggestion: Remove commit 536965e from this pull request (just in order to merge consistent things and not block other subsequent pull requests from being merged) and postpone updating the copyright headers to a later pull request. Then we should carefully fix all the copyright headers... I guess, we should discuss this personally...

I removed the commit for now. We need to discuss this especially with respect to the addition of a license in #90.

bockthom · 2018-02-18T14:45:11Z

tests/test-split.R

+    net.conf$update.values(list(author.relation = "cochange", simplify = FALSE))
+
+    ## retrieve project data and network builder
+    project.data = ProjectData$new(proj.conf)


The comment seems not to be up to date as the network builder is not retrieved here...

bockthom · 2018-02-18T15:27:57Z

util-networks-covariates.R

+#' @param aggregation.level Determines the data to use for the attribute calculation.
+#'                          One of \code{"range"}, \code{"cumulative"}, \code{"all.ranges"},
+#'                          \code{"project.cumulative"}, \code{"project.all.ranges"}, and
+#'                          \code{"complete"}. See a\code{split.data.by.networks} for


Typo in "a\code{split.data.by.networks}".
Also affects all the functions below...

bockthom · 2018-02-18T15:48:12Z

util-networks-covariates.R

+    return(nets.with.attr)
+}
+
+#' Add author role attribute


Maybe start a new subsection here for a better a structure of the file?
## * Role --------------------------------------------------------------

bockthom · 2018-02-18T16:05:31Z

util-networks-covariates.R

+#'
+#' @param list.of.networks The network list
+#' @param project.data The project data
+#' @param name The attribute name to add [default: "commit.count"]


Wrong default value here. It should be commit.count.author.not.committer.

bockthom · 2018-02-18T16:09:37Z

util-networks-covariates.R

+#'
+#' @param list.of.networks The network list
+#' @param project.data The project data
+#' @param name The attribute name to add [default: "commit.count"]


Also wrong default value here. It should be commit.count.committer.not.author.

bockthom · 2018-02-18T16:17:16Z

util-networks-covariates.R

+#' @param name.column The name of the author or committer column
+#'
+#' @return A list of networks with the added attribute
+add.vertex.attribute.commit.count = function(list.of.networks, project.data, name = "commit.count",


Just a short comment here regarding the name of the helper function: Currently, a non-expert user of the network library may use this function as the default function for commit count attributes, as the name of this functions sounds reasonable, instead of add.vertex.attribute.commit.count.author as auto-completion may also suggest add.vertex.attribute.commit.count in the first place. Are there any suggestions for a better name of the helper function to not be confused with the actual function to use?
(I would like to have this function as a private function if we would have dealt with classes here... - just to make my intention clear.)

I just renamed it to add.vertex.attribute.commit.count.helper. This should be enough for now.

Would we have a proper R package, this would be easy to hide. 😉

bockthom · 2018-02-18T16:20:34Z

util-networks-covariates.R

+#' @param default.value The default value to add if a vertex has no matching value [default: 0]
+#'
+#' @return A list of networks with the added attribute
+add.vertex.attribute.commit.count.committer = function(list.of.networks, project.data, name = "commit.count",


What about changing the default name here (and also in the documentation) to commit.count.committer?
In the end, only the usual commit count -- which is the author commit count -- should have the default name commit.count -- or would you also prefer to rename that to commit.count.author then for consistency reasons?

As we talk mainly about authors for now, we should withhold from renaming until we tackle #84.

bockthom · 2018-02-18T16:27:30Z

util-networks-covariates.R

+    return(nets.with.attr)
+}
+
+


While reviewing this pull request, I hit on the (crazy) idea to perform author-role classification not based on author networks or author commit count, but on committer networks or committer commit count.
I know, it's getting freaky now, but I will add this idea to issues #84 or #92 soon to just not forget the idea.

@bockthom

This patch introduces only some minor fixes and inprovements to the documentation for the changes in PR #93. The most important change is the renaming of the function 'add.vertex.attribute.commit.count' to 'add.vertex.attribute.commit.count.helper', mainly for unambiguity and consistency reasons. Props to @bockthom for being that precise. Signed-off-by: Claus Hunsen <[email protected]>

Felix Prasse and others added 30 commits December 21, 2017 17:10

Add commit count as vertex attribute

3ca6063

Add methods required to add vertex attributes to the graph. Add method to calculate commit count as vertex attribute Signed-off-by: Felix Prasse <[email protected]>

Add further vertex attributes

cafa3ab

Signed-off-by: Felix Prasse <[email protected]>

Add author classification attribute

32944cb

Signed-off-by: Felix Prasse <[email protected]>

Minor refactoring on vertex attributes

fe2721c

Signed-off-by: Felix Prasse <[email protected]>

Fix implementation of active author range vertex attribute

0371351

Signed-off-by: Felix Prasse <[email protected]>

Rename util-covariates.R to util-networks-covariates.R

39065fd

Signed-off-by: Felix Prasse <[email protected]>

Rename what parameter to aggregation

e3c6510

Signed-off-by: Felix Prasse <[email protected]>

Add comments to vertex attribute methods

cd261ad

Signed-off-by: Felix Prasse <[email protected]>

Extend add.vertex.attribute.first.activity to work on mail, commits a…

288c927

…nd issues Signed-off-by: Felix Prasse <[email protected]>

Add default value for missing vertex attributes

d7d8986

Signed-off-by: Felix Prasse <[email protected]>

Improve author role classification

2549f2b

Signed-off-by: Felix Prasse <[email protected]>

Add explicit return

bca66be

Signed-off-by: Felix Prasse <[email protected]>

Add test for splitting project data

efd9b2d

Signed-off-by: Felix Prasse <[email protected]>

Fix network attribute unit tests

df52ac7

The tests were referencing to a wrong project data variable Signed-off-by: Felix Prasse <[email protected]>

Fix artifact count attribute implementation

8ae1abf

Signed-off-by: Felix Prasse <[email protected]>

Add tests for network vertex attributes

b9f0bd3

Add test for add.vertex.attribute.author.role.simple Add test for add.vertex.attribute.active.ranges Signed-off-by: Felix Prasse <[email protected]>

Comments for network-covariates tests

3e20fb2

Signed-off-by: Felix Prasse <[email protected]>

Add more vertex attributes

9f53875

Add artifact attributes Add commit count based on committer Signed-off-by: Felix Prasse <[email protected]>

Add tests for artifact vertex attributes

a002dfa

Signed-off-by: Felix Prasse <[email protected]>

Update core-peripheral module

c6418c5

With respect to PR #67, logging statements, line breaks, and capitalization in sqldf-statements are updated. Signed-off-by: Claus Hunsen <[email protected]>

Update networks-covariates module

0eafcd0

With respect to PR #67, indentation, line breaks, coding flaws, and anything else is tackled. Adds a FIXME item regarding the first activity of an author, which will be tackled in an upcoming commit. Signed-off-by: Claus Hunsen <[email protected]>

Fix section dividers in some files

ac678f6

The section dividers always end at column 77. ;) Signed-off-by: Claus Hunsen <[email protected]>

Add additional FIXME items for the 'ProjectData$get.artifacts' problem

07c85db

Signed-off-by: Claus Hunsen <[email protected]>

Add 'date.offset' to possible edge attributes

0c1868a

Signed-off-by: Claus Hunsen <[email protected]> Signed-off-by: Thomas Bock <[email protected]>

clhunsen added 2 commits February 16, 2018 09:56

Move 'get.commit.data' to the data module

4810ac5

For better modularity, the function 'get.commit.data' is moved to the data module, right where it obviously belongs. Signed-off-by: Claus Hunsen <[email protected]>

Add example on vertex attributes to showcase file

056cd22

This code originates from @flx5's PR on vertex attributes (#67). Signed-off-by: Claus Hunsen <[email protected]>

clhunsen added the enhancement label Feb 16, 2018

clhunsen added this to the v3.1 milestone Feb 16, 2018

clhunsen mentioned this pull request Feb 16, 2018

Add vertex attributes #67

Closed

clhunsen changed the title ~~Add vertex attributes~~ Add vertex attributes (updated) Feb 16, 2018

This was referenced Feb 16, 2018

Possibility to build cumulative ranges #61

Closed

Use lubridate package for all POSIXct conversions #85

Closed

bockthom requested changes Feb 18, 2018

View reviewed changes

bockthom mentioned this pull request Feb 18, 2018

Correctly incorporate committer data everywhere #84

Open

4 tasks

bockthom approved these changes Feb 19, 2018

View reviewed changes

Merge branch 'dev' into vertex-attributes

253433d

clhunsen merged commit 5e677d2 into se-sic:dev Feb 19, 2018

clhunsen deleted the vertex-attributes branch February 27, 2018 08:13

clhunsen mentioned this pull request Mar 1, 2018

v3.1 #101

Merged

bockthom mentioned this pull request Mar 8, 2018

Improve performance for vertex attribute 'author.role' & fix default aggregation levels #102

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add vertex attributes (updated) #93

Add vertex attributes (updated) #93

clhunsen commented Feb 16, 2018 •

edited

Loading

bockthom left a comment •

edited

Loading

bockthom Feb 18, 2018

bockthom Feb 18, 2018

bockthom Feb 18, 2018

clhunsen Feb 19, 2018

bockthom Feb 18, 2018

bockthom Feb 18, 2018

bockthom Feb 18, 2018

bockthom Feb 18, 2018

bockthom Feb 18, 2018

bockthom Feb 18, 2018

clhunsen Feb 19, 2018

bockthom Feb 18, 2018

clhunsen Feb 19, 2018

bockthom Feb 18, 2018

Add vertex attributes (updated) #93

Add vertex attributes (updated) #93

Conversation

clhunsen commented Feb 16, 2018 • edited Loading

bockthom left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

clhunsen commented Feb 16, 2018 •

edited

Loading

bockthom left a comment •

edited

Loading