Added BUSpark notebook to forked repo.

[parkerwstone]

Related Issues and Dependencies

…

This introduces a breaking change

• Yes
• No

This Pull Request implements

… Explain your changes.

Description

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[sesheta]

Hi @parkerwstone. Thanks for your PR.

I'm waiting for a aicoe-aiops member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

Shreyanand commented on 2021-03-26T01:46:41Z
----------------------------------------------------------------

• Don't print really long outputs. It makes reading the notebook very difficult. Print only a few lines.
• Remove commented out code
• Use Markdown cells to define headings and sections (https://www.datacamp.com/community/tutorials/markdown-in-jupyter-notebook)
• Where is the ignoreWaiting function called? If not, remove it.

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

Shreyanand commented on 2021-03-26T01:46:41Z
----------------------------------------------------------------

• How is this different from the above cell, are we pulling the data twice?
• If it is repeated, remove it.

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

Shreyanand commented on 2021-03-26T01:46:42Z
----------------------------------------------------------------

Is this cell part of the analysis?

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

Shreyanand commented on 2021-03-26T01:46:43Z
----------------------------------------------------------------

• Again, please don't print everything, just print a few lines.
• Write a line about what you mean by "cluster"

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

Shreyanand commented on 2021-03-26T01:46:43Z
----------------------------------------------------------------

• Place the pip installs in the beginning of this notebook. Drain has already been imported in the previous cell, it wouldn't work if it is installed later. One of the goals while writing code is to make it reusable.

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

Shreyanand commented on 2021-03-26T01:46:44Z
----------------------------------------------------------------

• What is this cell telling us? How should we interpret the results?
• How is this different from the previous cell that prints clusters?
• Add explanation.

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

Shreyanand commented on 2021-03-26T01:46:45Z
----------------------------------------------------------------

Is the rest of this notebook old code? If yes, remove it, use this notebook only for code that you want to publish.

If the following cells are preprocessing steps or prelim. analysis then add it in the beginning of this notebook before the drain code. In that case, add interpretation of the analysis (word based analysis of logs suggests that ....)

[Shreyanand]

Hey guys! In the current state, the notebook has a lot of noise in it. Please use the comments to clean and structure it. The drain bit looks promising, let's work on understanding and presenting it in detail.

[Shreyanand]

The changes are a good start, but the notebook is still incoherent. Take this notebook for example, it starts with a heading and tells us what to expect in the notebook. The cells are separated and connected based on logical steps or sections. Each section has markdown associated with it explaining what to expect in the code next. Following this format, the notebook should become more clear.
One of the major thing to focus at this point is to add explanations on the output of the drain parsing. Importing and applying is the easy part, understanding and dissecting the results would require more time and effort. What does the current output mean? I think coming up with log examples to show how and when this method works would help.

[Shreyanand]

Some suggestions:

• If there are 256 logs, training with 100 and testing with 156 may not be enough for the model to train. 80% training and 20% test with StratifiedShuffleSplit split should give better results.
• Try xgboost:

from xgboost import XGBClassifier
XGBClassifier().fit(X_train, y_train)

Reply via ReviewNB

[sesheta]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign michaelclifford after the PR has been reviewed.
You can assign the PR to them by writing /assign @michaelclifford in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment