Hi,
I was thinking about adding a new option to pbs_benchpress called “skip-teardown”. I’ve been using “stop-on-failure” a lot, but I keep wanting it to not run teardown so that the system is left in the state right after the failure/error occurred, which is prefect for further debugging.
What does the community think about adding a new option? If I get no thumbs down then I’ll consider it an approval 
Thanks.
I’ve also wanted something like this, but more like “abort-on-failure” that would both skip the tear down and stop running tests when a failure is encountered so we can investigate the state of things. I think you may also need a “teardown-only” option you can run after you’re done investigating, but perhaps not.
Thanks for your inputs Mike. Initially I felt that “stop-on-failure” should also skip teardown, but there might be cases where that’s not desirable. But if everyone feels that “stop-on-failure” should also do “skip-teardown”, then we can just enhance “stop-on-failure” instead of adding a new option.
About “teardown-only”, I can see how that would help, but I haven’t felt the need for it personally. For now, I overwrite tearDown() to not do anything and that accomplishes “skip-teardown”, then I manually investigate what’s happening, fix my test or the code, and just run the test again and that cleans up the crud that was left behind earlier.
Thanks Ravi. I like the idea of skip-teardown. And I would vote for keeping it separate than stop-on-failure.
I think stop-on-failure should also skip teardown to help in debugging and keeping options simple.
Anamika in which case will it help if we have these options separate?
Hey @arungrover, it is helpful for the cases where part of the tests has to be run manually so you would want to preserve the setup. Also it was once discussed in the client meeting where they want to run some tests in production but with setUp and (specially) teardown off.
I think I was not clear enough. Here is what I would like to see.
Thanks for the explanation Anamika. I also agree that skip-teardown might have value by itself. But I am not sure if stop-on-failure without skip-teardown has any value or not. So, what if we added a new option “skip-teardown” and also enhanced “stop-on-failure” to skip teardown by default? Can you think of a case where the latter would be undesirable?
Ok I apologize, that’s exactly what you suggested, so I guess that’s the right way to go.
I think “stop-on-failure” should skip teardown. This will help in debugging and will be more widely used scenario.
pbs_benchpress’s -p option has ‘skip-teardown’ we should make use of this instead of adding direct option to pbs_benchpress.
hmm, I personally think that skip-teardown should be a direct option to pbs_benchpress, I can see it being more frequently used than some of the other direct options to pbs_benchpress (especially since we are talking about moving most of the actual teardown to tearDown instead of setUp in the future). But I’ll go with consensus, @arungrover and @anamika, do you guys have any opinion on this?
Hi @agrawalravi90 and @kjakkali, I don’t have any preference as long as it satisfies all the requirements. I am just worried that -p options are mostly unknown and over loaded which is different.
If we have a skip-teardown, should we also have the opposite? skip-setup?
Interesting thought … I can’t think of a use case though … But it does seems to go hand in hand with skip teardown …
@bhroam : Yes.pbs_benchpress’s -p option has ‘skip-setup’ option.
Hey @agrawalravi90, I would prefer if we reuse the existing option itself, especially when we also expose ski-setup using the same option. It will keep things consistent. That being said, we also need to improve on documenting all the options that can be given with ‘-p’ (at the very least pbs_benchpress’s usage should talk about these options)
@arungrover : Anne is updating pbs_benchpress command doc page.
Thanks for the pointer Kumar. This document will be really helpful to know what options can be passed to benchpress command. But, I also think we should update command’s usage because not everyone will go and look for the document on confluence. In case of commands it is really handy either to have an extensive usage or a man page.
Should this command become supported, with a man page?
I’m not sure about having it supported, but a man page would be helpful
Here’s something you can use: https://pbspro.atlassian.net/wiki/spaces/DG/pages/504627244/Using+pbs+benchpress+under+construction
Please feel free to help make it better.