Feature discoverable by pattern parent folder issue

[jonathanclausen]

Hi,

I think I have found an issue with the newly added feature for finding feature files by regex pattern #130. When I try to locate a feature file in a parent directory it is not being discovered.

For example, I have a file called fields.feature.

Referencing it as:
[FeatureFile(@"../fields.feature")] works just fine, however,
[FeatureFile(@"../*.feature")] is not shown in test explorer.

[ttutisani]

Just to clarify - the new feature was not yet released in NuGet. Did you download the code from GitHub and found the bug there?

[jonathanclausen]

Yes, I have cloned the repo and referenced it directly in my solution.

[miguelius]

Hi, Jonathan!

I've noticed it's the repository GetFeatureFilePaths method, which starts it's feature files look up from the current folder.

I'll fix it in the forked repo and if it's fine and Tengiz finds it proper I'll make a PR.

[miguelius]

Done! You can check it out in https://github.com/miguelius/Xunit.Gherkin.Quick!

[ttutisani]

I've noticed it's the repository GetFeatureFilePaths method, which starts it's feature files look up from the current folder.

I am not sure about this statement. The feature file repository looks up from the output directory, which is the root folder. What is not right about it?

[miguelius]

https://github.com/ttutisani/Xunit.Gherkin.Quick/blob/ee62ee35cec8067ad288116bf9530bf2c7d144c9/source/Xunit.Gherkin.Quick/CoreModel/FeatureFile/FeatureFileRepository.cs#L42 Well, Since the feature look up is based on the current folder and then filetered by matching. I can't never match what I doesn't gets to the matchet. Current way: - Find feature files from current dir - Return to the discoverer those who match the pattern. In the forked repository there is a simple proposed solution, which: - Starts the lookup from the base directory of the batterb - Fillters using the pattern with the Directory.EnumerateFiles API The matcher is left to be used by the MissingFeature discovery.

[ttutisani]

That line you referenced looks in the current dir, yes, but that is the current dir of the DLL, not the feature class, if that's what you think it is.

So the current dir of the DLL is the root dir of the output.

Starts the lookup from the base directory of the batterb

Do you mean "pattern"? What I see is your code looks at the path passed into it as an argument. That can be anything, even outside of the output folder. I don't think that is what the framework should allow doing.

Is that the intent here? To find the feature file outside of the output directory? In that case, I don't think that feature should be implemented. Not at this time.

[miguelius]

Actually looks for the current working directory, by default the dll is in the same directory. But it may not be and find the paths according to the process current directory. In our case we have the run it that way to separate the feature files from the dlls. We only distribute the dlls. The runner finds them. That's the point. Is this the intent. Perhaps in a xunit test project with code it wouldn"t be necessary. But running it from an independent runner, it lets the user count with their feature files in a different dir. Of course it's your decision. Including some sort of dynamic route for feature files would be nice. But it s all about scope!

[ttutisani]

What you described is not how it works, at least not on my machine. I just executed tests from another folder outside of the output directory, and it ran exactly the same way as if I executed it from inside the output folder. So the runner was not in the output directory and it runs fine. All the feature files are located without a problem.

I may be mistaken but I think there is a misunderstanding of what Jonathan wants, and your code does something else. Either way, I need a better understanding and a justification to review such a change.

[ttutisani]

@jonathanclausen please provide more details, including entire code for feature class and file, as well as where they are located exactly in your project. i.e., project file structure. I need to understand what you are trying to do exactly.

[jonathanclausen]

Hi,

I was placing feature files outside of the output folder, trying to match the Gherkin feature file relative to the output folder.

I have the following small test feature class

[FeatureFile(@"../features/*.feature")]
  public class StepsCore : Feature
  {
    [Given(@"field {string} is {string}")]
    public void FieldIs(string field, string value)
    {
      field.Should().Be(value);
    }
  }

The pattern "../features/*.feature" I would expect to 1) go up one level from the current directory, then, 2) go to "features" directory and then, 3) select all files ending in ".feature", thus matching the regular expression.

What I am really trying to accomplish is to seperate Gherkin feature files from the implementation. When one is able to specify a path to .feature files, I would assume (as a user of the framework) that any path is valid, not just paths in the output directory and subdirectories thereof.

But again, it is up to @ttutisani to decide what is within scope of the framwork. I have worked around the problem, by creating scripts that copy the feature files to the output directory of my application.

The directory structure is as follows:

TestRunner
  - bin
    - Debug
       - net5.0
          - TestRunner.dll
       - features
          - first.feature
          - second.feature
  - StepsCore.cs
  - TestRunner.sln
  - TestRunner.csproj

What I ultimately is looking to do is really to publish the TestRunner project without Gherkin feature files, then let users place feature files in predefined feature folder relative to the TestRunner project, like so:

- TestRunner (published app)
   - TestRunner.dll
   - ...
- Features
   - first.feature
   - second.feature
   - ...

The FeatureFile attribute of the StepsCore class would then need to be "../Features/*.feature"
Thus, having completely seperated feature files and application files.

[ttutisani]

Thanks for clarifying. I don't think that it is easy to implement patterns entirely without issues, and the more we will implement, the more bugs there will be. So for now, I will be limiting the placement of files to the output directory if you use the pattern.

[jonathanclausen]

Okay, fine by me.

While I have you guys here, there is something with this new feature that should be looked into. When I use a regex to find feature files like so: [FeatureFile("*.feature")] all the scenarios in the test explorer are grouped under the same Feature.

Have a look here: https://imgur.com/a/r7xQggY

Shouldn't they be separated by Feature as they are without the regex?

[ttutisani]

Your picture shows them separated by feature. First several belong to the "Approve Application", next several belong to the "CompensationApproval", and so on. I don't understand what you expect differently.

[ttutisani]

FYI - I just changed the implementation to explicitly handle regex. The above-mentioned implementation replaced the asterisk with the regex pattern and then used regex matching against the file paths. That was confusing and would lead to errors such as accidental regex in the path or the opposite (regex being messed up). So instead, the implementation now supports regex and the asterisk will be considered as part of the regex itself and won't be replaced.

The attribute FeatureFile takes the second parameter which indicates whether the path is a regex. The default is a simple path (not regex) for backward compatibility. You will need to make this change to your tests if they used the attribute with a pattern before.

Just letting you know in case you get the latest and something stops working or works differently.

With the current implementation, I feel comfortable moving forward and releasing shortly.