Currently test_resource_create and test_resource_delete are marked as smoke tests in PTL but I think those are not smoke test instead its regression test cases for qmgr resource interface as it run every possible combination of resource creation and deletion using qmgr.
So, I propose to remove test_resource_create and test_resource_delete from smoke tests, move it under regression and add just smoke test for qmgr interface.
Can you please explain a bit more about what exactly the test is doing? every possible combination of resource creation and deletion meaning what? Is it trying to create and delete all different types of resources? all different flags?
I personally think that the tests should be “trimmed” if they are too exhaustive instead of completely removing them as smoke tests. I think it’s good to know that one can create and delete a resource as part of build testing. So, if they are too exhaustive to be smoke tests, then maybe we should create thinner versions of them for smoke testing. Or have the tests conditionally be less exhaustive when run as smoke tests (not sure if that’s possible, just food for thought).
Yes @agrawalravi90 Test run every possible combination of different types with different flags.
May be there is some misunderstanding here, I am not saying that we should test cases for testing resource creation and deletion from smoke tests. What I proposing is current tests doesn’t full regression testing of resource creation and deletion using qmgr, so move those test cases to regression and add new tests in smoke tests which will do minimal required test of resource creation and deletion using qmgr interface.
Hope you got my point.
Thanks for clarifying, your proposal sounds good to me.
Hi @hirenvadalia, as long as we are adding smoketest for creating and adding all types of supported custom resources this proposal looks good. May be we should add those test first before removing these tests.
Just adding that test_resource_delete takes approx ~3min and test_resource_create takes ~20s.
I agree that these tests are a bit excessive for a smoke test. I’ve always thought so. I do like the idea of slimming the tests down to practically nothing for a smoke test.
I personally think we should make a review of all the smoke tests. The last time I took a brief look, there was one that just tested PTL functionality.
Thanks @agrawalravi90, @anamika and @bhroam for response.
@anamika Rather than adding new test first and moving current tests separately, we can do it together. Please let me know if you think otherwise.
Anyways, I will create a ticket for same.