Testing Source Distributon

[jgcode]

[Cross posted to LAVA]

 

Hi

 

I made a post to LAVA, which arises from using VIPM Pro to build packages.

I want to put together a test suite and have a question before I start:

 

I am interested to know what is the standard practice for testing source distribution code (i.e source that is only used for create a source distribution):

Should I write test-cases/perform-testing for/on source code OR source distribution code OR both?

 

The reason I ask is that I can see issues with either choice:

Source only - you want to test source to pick up any errors before you make it into a source distribution but...

Distribution only - distribution may apply name changes and in case of an error with dependency code that meant the source was ok but the distribution isn't - do I want to test this too?

Both - best coverage, but redundant tests case, changes mean updating code (tests) in two places (due to re-naming etc..)! Should I still run manual tests on the package? Or is there a way to automate this?

 

What do you guys do?

 

Cheers :beer:

 

JG

[Omar Mussa]

I am interested to know what is the standard practice for testing source distribution code (i.e source that is only used for create a source distribution):

Should I write test-cases/perform-testing for/on source code OR source distribution code OR both?

What do you guys do?

 

Excellent questions! Thanks for using VI Tester and VIPM Professional. We are really excited by these kind of challenges! I actually haven't had to create source distributions in a while, but we have the exact same problems you are talking about when it comes to our product development for products like EasyXML, etc.

 

Here is our general approach:

1) ALWAYS test the source code itself - testing the source code is a pre-requisite to shipping your distribution because it is the easiest way to know if your code is 'ship-able'. If your source tests fail, you have issues that you need to look into. This will actually save you time in the long run because you won't waste your time on the build step for source that is 'broken' (meaning failing to meet the functional requirements that you are unit testing).

 

2) Test the 'critical' aspects of your source distribution/build process. By critical, I mean, testing components that may have undergone changes while building your source distribution. This can be as trivial as creating tests that validate that certain VIs are password protected, etc. It can be tricky, depending on how complex your source distribution process is, but your goal should probably be to test that the process 'worked'.

 

Your post made me think of another way to do this... I promise to write a blog article about it though because its too complex and too much info to put into this post.

[jgcode]

Excellent questions! Thanks for using VI Tester and VIPM Professional. We are really excited by these kind of challenges! I actually haven't had to create source distributions in a while, but we have the exact same problems you are talking about when it comes to our product development for products like EasyXML, etc.

 

Here is our general approach:

1) ALWAYS test the source code itself - testing the source code is a pre-requisite to shipping your distribution because it is the easiest way to know if your code is 'ship-able'. If your source tests fail, you have issues that you need to look into. This will actually save you time in the long run because you won't waste your time on the build step for source that is 'broken' (meaning failing to meet the functional requirements that you are unit testing).

 

2) Test the 'critical' aspects of your source distribution/build process. By critical, I mean, testing components that may have undergone changes while building your source distribution. This can be as trivial as creating tests that validate that certain VIs are password protected, etc. It can be tricky, depending on how complex your source distribution process is, but your goal should probably be to test that the process 'worked'.

 

Your post made me think of another way to do this... I promise to write a blog article about it though because its too complex and too much info to put into this post.

 

Thanks for the reply Omar. I originally thought it may be a stupid question to ask, but I just couldn't get my head around it. I really look forward to your blog post on this. I just finished coding my automated build script based on Justin's blog entry in February - !Awesome!