"Invalid tree info" - Fileless directories #259

blast-hardcheese · 2019-07-13T04:50:06Z

Following along with sbt-microsite: Publish using GitHub4s, I've created a token with the repo scope, and have Enable SSO'd (which is required by my environment).

I was then able to use the Github API to perform some simple tests:

List commits for my repo
List entries for a given tree

Despite this, when attempting to use publishMicrosite, I get:

[error] Error committing filesGitHub returned an error: Failed invoking with status : 422 body :
 {"message":"Invalid tree info","documentation_url":"https://developer.github.com/v3/git/trees/#create-a-tree"}

        at sbtorgpolicies.github.syntax$GHResponseOps.execE(syntax.scala:53)
        at sbtorgpolicies.github.syntax$EitherTOps.execE(syntax.scala:43)
        at sbtorgpolicies.github.GitHubOps.commitDir(GitHubOps.scala:172)
        at sbtorgpolicies.github.GitHubOps.commitDir(GitHubOps.scala:148)
        at microsites.MicrositeAutoImportSettings.$anonfun$micrositeTasksSettings$13(MicrositeKeys.scala:305)
        at scala.Function1.$anonfun$compose$1(Function1.scala:44)
        at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:40)
        at sbt.std.Transform$$anon$4.work(System.scala:67)
        at sbt.Execute.$anonfun$submit$2(Execute.scala:269)
        at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
        at sbt.Execute.work(Execute.scala:278)
        at sbt.Execute.$anonfun$submit$1(Execute.scala:269)
        at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
        at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: github4s.GithubResponses$UnsuccessfulHttpRequest: Failed invoking with status : 422 body :
 {"message":"Invalid tree info","documentation_url":"https://developer.github.com/v3/git/trees/#create-a-tree"}
        at github4s.HttpRequestBuilderExtensionJVM.toEntity(HttpRequestBuilderExtensionJVM.scala:84)
        at github4s.HttpRequestBuilderExtensionJVM.toEntity$(HttpRequestBuilderExtensionJVM.scala:74)
        at github4s.jvm.Implicits$.toEntity(Implicits.scala:21)
        at github4s.HttpRequestBuilderExtensionJVM$$anon$1.$anonfun$runMap$3(HttpRequestBuilderExtensionJVM.scala:67)
        at scala.util.Try$.apply(Try.scala:209)
        at sbtorgpolicies.github.instances$$anon$2.capture(instances.scala:57)
        at sbtorgpolicies.github.instances$$anon$2.capture(instances.scala:56)
        at github4s.HttpRequestBuilderExtensionJVM$$anon$1.runMap(HttpRequestBuilderExtensionJVM.scala:61)
        at github4s.HttpRequestBuilderExtensionJVM$$anon$1.run(HttpRequestBuilderExtensionJVM.scala:34)
        at github4s.HttpClient.post(HttpClient.scala:159)
        at github4s.api.GitData.createTree(GitData.scala:251)
        at github4s.free.interpreters.Interpreters$$anon$9.$anonfun$apply$7(Interpreters.scala:295)
        at cats.data.Kleisli.$anonfun$map$1(Kleisli.scala:19)
        at cats.data.KleisliFlatMap.$anonfun$tailRecM$2(Kleisli.scala:534)
        at cats.instances.TryInstances$$anon$1.tailRecM(try.scala:57)
        at cats.instances.TryInstances$$anon$1.tailRecM(try.scala:15)
        at cats.data.KleisliFlatMap.$anonfun$tailRecM$1(Kleisli.scala:534)
        at cats.data.Kleisli.$anonfun$map$1(Kleisli.scala:19)
        at cats.data.KleisliFlatMap.$anonfun$tailRecM$2(Kleisli.scala:534)
        at cats.instances.TryInstances$$anon$1.tailRecM(try.scala:57)
        at cats.instances.TryInstances$$anon$1.tailRecM(try.scala:15)
        at cats.data.KleisliFlatMap.$anonfun$tailRecM$1(Kleisli.scala:534)
        at github4s.Github$GithubIOSyntaxEither.exec(Github.scala:62)
        at sbtorgpolicies.github.syntax$GHResponseOps.execE(syntax.scala:51)
        ... 19 more

I found 47degrees/sbt-org-policies#388 (comment), which was largely unhelpful other than to suggest that perhaps despite supplying the token, it doesn't work as advertised (possibly in my SSO environment).

Any assistance would be appreciated!

The text was updated successfully, but these errors were encountered:

bilki · 2019-07-15T09:54:38Z

Hi @blast-hardcheese! Is your microsite repo initialized? Also double check that your sbt values micrositeGithubOwner and micrositeGithubRepo are correct.

blast-hardcheese · 2019-07-16T05:49:25Z

@bilki I presume so. It would be easier to just link where I am at this point.

I was successfully able to publish to a non-SSO-protected repo with the API token, fwiw.

bilki · 2019-07-17T15:46:39Z

Hey, @blast-hardcheese would you mind (if possible) trying to publish the microsite to another clean empty branch, just to be sure it's not the content of gh-pages that is provoking this bug?

blast-hardcheese · 2019-07-19T06:19:55Z

@bilki OK, so after digging quite a bit, I discovered that the root problem here is I'm getting NewTreeRequest(Some(...), Nil), as I have a tree containing only trees.

I've narrowed down what I suspect is the culprit: def getAllFiles: List[File] = Option(dirToCommit.listFiles()).toList.flatten.filter(_.isFile).

It seems as though we're doing top-down to create all the directories, but I suspect github really wants us to navigate depth first, creating the trees as the full hierarchy.

I'm wrapping up for the night, but this is where I got. If you've got some advice on how to proceed, (is flipping processing from top-down to bottom-up acceptable to you?) I'd appreciate it.

Thanks so far.

bilki · 2019-07-23T14:14:48Z

Hey @blast-hardcheese, could you check if your site folder contains any empty dir (in the machine from where you are trying to publish)? I get the exact same error Invalid tree info when I try to commitDir a folder that contains empty dirs. As soon as I remove them, it works like a charm.

blast-hardcheese · 2019-07-24T16:19:19Z

@bilki I have a directory that is empty, save for other directories, which contain files. The problem, as I've seen it, is that we create trees only containing blobs, then go back and create the next deepest tree with those blobs, with the parent being another tree.

My proposal would be to do a depth-first approach, committing the deepest tree, then collecting that tree's hash as another sibling in the next highest up tree to generate.

What do you think?

stanleydunne · 2019-07-26T11:26:49Z

I am seeing the same error "Invalid tree info" when trying to use the API to add individual files to my repo. I don't have empty folders.
Has a solution been found?
Edit : I found other issue in my own code that was corrupting the data I was passing in. Ignore my question.

bilki · 2019-08-08T11:01:18Z

Sorry about the delay @blast-hardcheese. That sounds like a possible solution, have you already tried it? I honestly don't know in what order github expects dir creations to happen. Anyway, you could open an issue at https://github.com/47deg/sbt-org-policies/issues, which is where the order on how commit dirs are updated is decided, and it's where your suggestion will apply.

calvellido · 2019-09-19T10:27:12Z

I've been using github4s through sbt-microsites too, and after messing a bit with this, I can confirm that the error will happen with a project structure like this. Source directory (src/main):

resources▼
- microsite▼
  - data▼
    - menu.yml
scala▼
- HelloWorld.scala
tut▼
- docs▼
  - index.md
  - section-1-1.md
  - section-1-2.md
  - section-1.md
- index.md

Then, processed through sbt-microsites, this is the actual content that will be committed with github4s:

css►
docs►
highlight►
img►
js►
microsite▼
- data▼
  - menu.yml
_config.yml
index.html

I found that github4s will struggle to commit this structure (422), with the culprit being the fileless microsite directory. As a workaround I added a .gitkeep file near the data directory there and then github4s can proceed.

Somehow, a thing that indirectly solve this, is to use sbt-microsites in a different module, instead of the root one 🤷‍♂️ This solves it because using sbt-microsites that way, the microsite/data/menu.yml won't be generated (hence not committed), and the case is that it shouldn't (47degrees/sbt-microsites#335).

But, summarizing, and getting back to the bug, after some other tests, we can conclude that github4s will fail to commit a tree that include some kind of fileless directory like this (I don't think other considerations on the tree shape will matter really for this case):

static▼
- subdir▼
  - data.yml

Being subdir the only child of the static directory.

artemkorsakov · 2020-09-17T16:39:26Z

I have the same problem.
While I had such a structure the publication was successful:

dir1
- dir2
  - index.md
  - file1.md
  - file2.md
- index.md
- file1.md
- file2.md
index.md
file1.md
file2.md

When I got such a structure that the publication began to fail with an error "Invalid tree info":

dir1
- dir2
  - index.md
  - file1.md
  - file2.md
- dir3
  - dir4
    - index.md
    - file1.md
    - file2.md
- index.md
- file1.md
- file2.md
index.md
file1.md
file2.md

I got directory dir3 with no files in it - just another directory (dir4)
The error "Invalid tree info" went away only after I added the file to the directory dir3:

dir1
- dir2
  - index.md
  - file1.md
  - file2.md
- dir3
  - dir4
    - index.md
    - file1.md
    - file2.md
  - index.md
- index.md
- file1.md
- file2.md
index.md
file1.md
file2.md

calvellido changed the title ~~"Invalid tree info"~~ "Invalid tree info" - Fileless directories Sep 19, 2019

calvellido added the bug label Sep 19, 2019

calvellido mentioned this issue Sep 19, 2019

microsite dir being copied when using sbt-microsites in main sbt module 47degrees/sbt-microsites#335

Closed

47erbot removed the bug label Sep 17, 2020

calvellido added the bug Something isn't working label Sep 18, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"Invalid tree info" - Fileless directories #259

"Invalid tree info" - Fileless directories #259

blast-hardcheese commented Jul 13, 2019

bilki commented Jul 15, 2019

blast-hardcheese commented Jul 16, 2019

bilki commented Jul 17, 2019

blast-hardcheese commented Jul 19, 2019

bilki commented Jul 23, 2019

blast-hardcheese commented Jul 24, 2019

stanleydunne commented Jul 26, 2019 •

edited

Loading

bilki commented Aug 8, 2019

calvellido commented Sep 19, 2019 •

edited

Loading

artemkorsakov commented Sep 17, 2020

"Invalid tree info" - Fileless directories #259

"Invalid tree info" - Fileless directories #259

Comments

blast-hardcheese commented Jul 13, 2019

bilki commented Jul 15, 2019

blast-hardcheese commented Jul 16, 2019

bilki commented Jul 17, 2019

blast-hardcheese commented Jul 19, 2019

bilki commented Jul 23, 2019

blast-hardcheese commented Jul 24, 2019

stanleydunne commented Jul 26, 2019 • edited Loading

bilki commented Aug 8, 2019

calvellido commented Sep 19, 2019 • edited Loading

artemkorsakov commented Sep 17, 2020

stanleydunne commented Jul 26, 2019 •

edited

Loading

calvellido commented Sep 19, 2019 •

edited

Loading