How to speed up the test driven development (TDD) cycle

[Roald87]

When going TDD you 1) write a failing test 2) adjust your function/function block such that is passes the test and then go back to 1). A crucial part of this cycle is how fast you can get feedback. I notice that with large projects this cycle is currently very slow. What I have is the following:

• Stand alone PLC project (which also includes my tests).
• TwinCAT project which imports the stand alone project via a .tmc file.

What I do is either
Method A

1. Write my tests/code in the stand alone project.
2. Build the project, such that a .tmc file is created.
3. Active the project from the TwinCAT project.

or
Method B

1. Write my tests/code in the stand alone project.
2. Log in with online changes
3. Do a "Reset cold"
4. and "Start" the project

Method B is a bit faster (just shy of 1 minute), but not always possible. Method A is slower (1.5 minute) especially the build steps

Generate TMC information ...
Import symbol information ...

take quite long. Maybe I'm doing something stupid, but I was wondering if anyone has a better solution?

Possible solution: Hard link FBs

You make a new TwinCAT project which imports the Function block to test and its unit test. You don't copy them, but use a hard link to 'copy' them to your new project folder.

Pro: project builds faster

Con: If there are a lot of dependencies (structs and FBs/functions) which are used by the FB you want to test, all the dependencies also need to be imported.

[RogerChristopher]

I generally follow the rule that any POUs requiring unit tests should be in a library, and any POUs in a library should have a unit tests. So my libraries have a PLC that can run the unit tests for that library - which is fast.

I have a project that runs all the unit tests for all my libraries, so as a final check I can test that changes to a library have not broken anything in another library.

Where I need unit tests in a machine project, I have a compiler switch to disable code execution (but not initialisation), and enable units test defined in the project. I can then call test object instances and project object instances as necessary.

Putting POUs in libraries also adds a useful discipline of keeping dependancies to a minimum and making them explicit.

[Roald87]

Hi Roger,

Thanks for the reply. We started putting some things into libraries, but most of it is not currently. It would probably be a good idea, especially with regards to keeping the dependencies to a minimum as you mentioned. I'll think about that in the future.

I'm not sure if I understand the compiler switch. You do that with a conditional pragmas? Does this method really speed things up?

Roald.

[RogerChristopher]

Yes. I use it to inhibit execution of the code, but run the tests:

[sagatowski]

I too try to keep everything in separate libraries as far as it possibly makes sense. Each library has its owning running-program, that can be executed everytime something is changed in the library. To automate this further, we are using a separate program (TcUnit-Runner), that once a change is done in Git/Version control and using and automation server (Jenkins in my case) does:

1. Opens the project with the same version of visual studio/TwinCAT (using the DTE/Automation Interface) as the project was created in
2. Compiles the project
3. Finds all tasks and disables all except the unit-test task (in case there are more tasks)
4. Finds the unit-test task and executes it
5. Collects the test-results
6. Writes the results into a xUnit/JUnit compatible XML-file (that Jenkins understands)
   When you have a couple of dozen of projects it's nice to have this feature. It's quite neat to automate this as far as possible, so that we can detect a break in the software if some tests fails.

I'm releasing this TcUnit-Runner software as open-source quite soon, but still have tons of documentation left to do (which takes more time than the writing the actual software...)

On this same discussion and regarding linking, there is this discussion:
#93

[sagatowski]

By the way, I love these types of discussions. Thanks for creating these type of issues, they are my favorite :-)

[sagatowski]

Yes. I use it to inhibit execution of the code, but run the tests:

@RogerChristopher Such a great idea. In projects where I'm mixing "normal MAIN-programs" and unit tests, I've always put them in separate tasks (PlcTask & UnitTestTask for example). In this way it's possible to keep them both in the same and do it with conditional pragmas instead. Never thought of doing it this way. Neat.

[Roald87]

By the way, I love these types of discussions. Thanks for creating these type of issues, they are my favorite :-)

Thanks! I wasn't sure if it was ok to create one here, because it is not really an issue which can be closed. It is more of an ongoing discussion/collection of ideas. I also doubted to post it on Stackoverflow, but since it is quite specific to TcUnit I figured this was probably a more suitable place. Also there it would probably be closed as "Subjective" or something 😛

I sometimes find it difficult that the information on TwinCAT is so sparse. Luckily that is changing, with people like you @sagatowski, and others creating blogs and open source projects. Sometimes I feel that coding in TwinCAT now-a-days is what it must have felt like to code 15 years ago, before Stackoverflow. So with almost no additional information online, besides the official documentation and a random forum post here and there.

[sagatowski]

@Roald87 , it's more than OK - it's exactly these type of "issues" which make me rethink some things. Usually you take some things for granted, and don't think outside of your own narrow box. So when issues like these are created it often makes me think "aha... yes this is another way to do it!".

And yes, I fully share your thoughts on the software development. My background is C++, and coming from that to with an infinite amount of information on stackoverflow to the PLC-world where most of the information is Beckhoffs infosys was quite a disappointment. But another way to see it is that there is plenty of room for improvement :-)

[sagatowski]

@Roald87 - You gave me an idea to create the new label"Discussion"! 😃

[Roald87]

But another way to see it is that there is plenty of room for improvement :-)

That is true! Basically everything which is successful in other languages you can copy to TwinCAT!

Another unrelated question: How do you refer to the language used when coding in structured text? Saying it is made in IEC 61131-3 is quite hard to say 😄 .

[Aliazzzz]

Mostly it is reffered as coding in 'pure' IEC or coding in 'IEC ST' on the forums I visit.
So the 61131-3 is ommited ;-)

[RogerChristopher]

By the way, I love these types of discussions. Thanks for creating these type of issues, they are my favorite :-)

Me, too. I don't understand why Beckhoff don't have a user forum - big missed opportunity to develop a community and get valuable feedback.

[Aliazzzz]

You are heartly welcome on the forum I hang out though! They encourage every user of every brand implementation to be welcome. It is hosted by CODESYS (hint...), and since TwinCAT is an implementation of CODESYS, also binary compatible. Sure, Beckhoff has made some big changes in its branded implementation, but on code level it is all more or less the same (slightly oversimplified).

See you there.

[sagatowski]

@Roald87 & @Aliazzzz - I've always said it like "61131-3 ST", but I really like @Aliazzzz suggestion on "IEC ST", so I'm going to start saying that.

Even though there is a codesys user forum I still see a big point in having a separate TwinCAT-only forum as there are so many specifics/differences compared to codesys. Even though the basic PLC programming might not differ too much, there is quite a big difference in the different optional libraries. They also have some stuff which are purely developed independent of codesys, like the new TF7000 Vision, the TF2000 HMI etc. I see posts regarding TwinCAT on plctalk.net, but as this is an US forum, it's mostly other brands than Beckhoff.

[Roald87]

We could of course start our own TwinCAT forum. Although a forum is a bit messy.

Ideally you would have some combination of Github issues/wiki and Stack Overflow. So a system with bidirectional links like Github issues/stackoverflow, so different topics/questions/posts can be linked from two directions. An upvote system like Stack Overflow to encourage and filter good answers/posts. And a way to easily add/edit some examples/docs/discussion like a wiki/forum, but without all the restrictions stackoverflow has about posting a subjective question.

[RGrabichler]

I would love that!
I have so many Questions, some unrelated or not speicific enough, and therefore i am unsure where to put them.
With a Forum, that would be much simpler.
And i think, i am not the only novice with this Qs, so others can learn too.

[mosri]

Hello- been lurking this repo for a while and just wanted to drop in and say that I feel all of your pain for some sort of TwinCAT forum/community. I've been using TwinCAT for about 3 years and felt like I've been developing software in the 90s, but recently found out about https://alltwincat.com/, which has been a really great addition. If you haven't already, I'd suggest everyone sub to r/TwinCAT on Reddit. It is fairly active on there and we could use more members!

I would like to add TcUnit has been a great initiative and a nice refresh from something like OSCAT -- which is pretty good, but quite outdated. I think if the community becomes more organized and has a place for discussion, the more likely we'll be able start other open source projects with many more people that could contribute.

As for how one refers to IEC 61131-3... people in the industry I know (mostly TwinCAT people) normally refer to it as Structured Text or ST. I think that sounds best and sounds just like its just any other programming language.

Feel free to add me on LinkedIn- Linkedin.com/in/markmosri.

[Aliazzzz]

Hello there @mosri,
As far as I know, https://alltwincat.com/ is the blog from @sagatowski :-)

Also, I am very curious why you say the following: "I've been using TwinCAT for about 3 years and felt like I've been developing software in the 90s". Would you care to elaborate on this a bit?

Hoping to hear from you.

Aliazzz

[Roald87]

If you haven't already, I'd suggest everyone sub to r/TwinCAT on Reddit.

I did join r/plc some time back, but didn't look for r/TwinCAT. Thanks for the tip!

[mosri]

@Aliazzzz - Mhmm I was quite tired when I said that, but what I meant to say was that there aren't many open source projects or libraries. As opposed to python or javascript where there are tons of open source libraries/projects for various things, as well as a large community actively discussing things about TwinCAT. Not that TwinCAT itself is outdated or anything. Quite the contrary- as far as developing in TwinCAT, it is light years ahead of anything out there and similar to modern software development, especially with Visual Studio/ST+OOP/Git.

@Roald87 and no problem- look forward to seeing you in our discussions!

[Roald87]

@mosri It is indeed unfortunate that there are not so many open source projects. It makes it harder to learn.

[Aliazzzz]

Personally, I am now working on a really cool open source project on CForge in team effort.
It is a library for SparkplugB support written for CODESYS3 written in IEC-ST

When it is ready, we can rebuild it with minimal effort for TwinCAT3. Basicly we did it with TcUnit too. We forked that (in an open source team effort) into CfUnit for CODESYS. Maybe we can do SparplugB in reverso :-)

I expect a 95% same codebase for a TwinCAT variant, but some libs should be replaced with Beckhoff counterparts with maybe some code tweaking by consequence. After this, the lib should compile/run on TwinCAT without any issue's (I hope).
Anyway, first we need to finish a minimum viable product.

[sagatowski]

I would like to add TcUnit has been a great initiative and a nice refresh from something like OSCAT -- which is pretty good, but quite outdated. I think if the community becomes more organized and has a place for discussion, the more likely we'll be able start other open source projects with many more people that could contribute.

Regarding OSCAT there is another open-source initiative which seems to be a fork of OSCAT:
https://github.com/RWTH-EBC/AixOCAT

[Beidendorfer]

I too try to keep everything in separate libraries as far as it possibly makes sense. Each library has its owning running-program, that can be executed everytime something is changed in the library. To automate this further, we are using a separate program (TcUnit-Runner), that once a change is done in Git/Version control and using and automation server (Jenkins in my case) does:

1. Opens the project with the same version of visual studio/TwinCAT (using the DTE/Automation Interface) as the project was created in
2. Compiles the project
3. Finds all tasks and disables all except the unit-test task (in case there are more tasks)
4. Finds the unit-test task and executes it
5. Collects the test-results
6. Writes the results into a xUnit/JUnit compatible XML-file (that Jenkins understands)
   When you have a couple of dozen of projects it's nice to have this feature. It's quite neat to automate this as far as possible, so that we can detect a break in the software if some tests fails.

I'm releasing this TcUnit-Runner software as open-source quite soon, but still have tons of documentation left to do (which takes more time than the writing the actual software...)

On this same discussion and regarding linking, there is this discussion:
#93

Hi everybody,
first of all thank you very much for all the great work, really great
I have been following TC_Unit for a while now and would like to integrate it better into our programmes. I am very interested in how you do unit tests in a library or how you test the library

If you have the tests in a library or use a test soltuion where you test the different library FB's.

At the moment I'm not sure what the best way is.
I see 3 possibilities:

1. to create a test soltuion (which contains the tests and library)

2. the unit tests direct in the lib. e.g. a main programme which executes the unit test.

3. the unit test in the own application which is activated by a switch (if defined(Test) ) between Application and Unit Test

Have I missed a solution? what would you recommend?

Thanks

[Roald87]

@Beidendorfer You can also create symbolic links to your library files from a separate unit test project as explained here https://stackoverflow.com/a/60542846/6329629. Then you do not need to reinstall your library each time you make a change.

[Beidendorfer]

@Roald87 thank you for pointing that out. that with the re-install the lib is an argument. I have to take a closer look at the symbolic links.

i.e. you would prefer the first variant with symbolic links to the libary?

[Roald87]

@Beidendorfer I prefer option 2 if possible. If you use that then you need to mark all your test function blocks are INTERNAL, otherwise they are available to the end-user.

Edit 26.09.2020: Additionally you can add an {attribute 'hide'} before all unit test function blocks/programs, so that they do not show up in the library manager.

[sagatowski]

I too opt for option 2, which is what I've mostly used for different projects. Except for the obvious advantage of it being more modular, it's also easier to run the tests (as you don't have any other program/task that you need to disable/take care of). Might be good to know that you can also hide the usage of TcUnit in your libraries completely (so that the projects that have references to your libraries don't even see the TcUnit imports):

https://tcunit.org/frequently-asked-questions/?Display_FAQ=715

I've used this when I developed a library for a consortia that they needed.

[Beidendorfer]

Thanks guys, I'll give it a try. it's a good tip for best practice with TcUnit

[Bulkje]

My company also adds test directly in the library, all the test FB's are set to INTERNAL .
Maybe I'll make a fork of tcunit-runner that excludes the map with the tests so they are not compiled when making the library.

What I wanted to ask is how you deal with the "Restart TwinCAT System in Run Mode" dialog. Activating configuration is easy, but then I have to wait for the program to compile and then the popup comes up. This is really inhibiting the TDD flow. I get jealous when I hear developers that use other languages talk about TDD, their tests complete so fast! I know the TwinCAT powershell module has commands that start TwinCAT without confirmation dialogs, so maybe it's possible with Visual Studio?

I know TcUnit-Runner launches the DTE with the -Quiet flag, is it also possible to use this in our own projects? Then I can just click activate configuration and walk away to get coffee while the test suite completes.

Another point, for CI with TcUnit runner we use a plc that is connected over the internet. This in itself is not the problem, but when starting TwinCAT on the PLC, the connection drops temporarily so not all TcUnit results are logged in the error window. Is it possible to generate the test result file in TwinCAT itself, so it can be sent to TcUnit-Runner when there's a connection? The workaround I have now is delaying TcUnit.RUN() with a timer, so the connection can be restored before the results start coming in. This is not ideal for the developers themselves, who also have to wait for the timer, or change the code (and remember not to commit it).
Maybe compillerdefines also fix this.