# PP-484: PP-485: Enhance output of qstat to make it more admin and script friendly

**URL:** <https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344>\
**Category:** Developers\
**Created:** [November 14, 2016, 9:40am UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344 "2016-11-14T09:40:36Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![nithinj](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/nithinj/32/45_2.png) [@nithinj](https://community.openpbs.org/u/nithinj)\
**Post date:** [November 14, 2016, 9:40am UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/1 "2016-11-14T09:40:36Z")

</div>

Hi All,

I have posted an EDD ( [EDD for enhancing qstat output](https://pbspro.atlassian.net/wiki/display/PD/PP-484%3A+PP-485%3A+Enhance+output+of+qstat+to+make+it+more+admin+and+script+friendly) ) for the proposed changes to enhance the output of qstat to make it more admin and script friendly. Please review and provide feedback.

Thanks,  
Nithin.

---

<div class="post-metadata">

**Author:** ![scc](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/scc/32/234_2.png) [@scc](https://community.openpbs.org/u/scc)\
**Post date:** [November 14, 2016, 1:57pm UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/2 "2016-11-14T13:57:16Z")

</div>

Hello Nithin, thanks! I have a few comments:

1. Jon ended up performing much of this work previously for ticket PP-483, see [https://pbspro.atlassian.net/wiki/pages/viewpage.action?pageId=31621169](https://pbspro.atlassian.net/wiki/pages/viewpage.action?pageId=31621169). That should be either be used as the starting point or allowed to stan on its own, depending on what we decide in comment 2) below. I am sorry if I didn’t mention the existence of this EDD previously.

2. I believe that filing PP-508 as a user story was unnecessary. I had previously filed these tickets surrounding the planned qstat changes:

PP-483  
As a user or admin, I would like qstat to return a full job attribute list with 1 line per job attribute, so that I can more easily use tools like grep to find the information I am looking for regarding my jobs

PP-484  
As an admin or user, I would like qstat to be able to output full job attribute information in JSON format, so that I can more easily write scripts consuming this information

PP-485  
As an admin or user, I would like qstat to be able to output full job attribute information in delimeter separated value format, so that I can more easily write scripts consuming this information

We can either change PP-508 to be an epic that encompasses those three tickets, or close it altogether. Before I take any action like that, was one of those options your original intention? And if so, was your plan to add the JSON and DSV interface options to this same EDD page?

1. About Interface 1 handling Resource\_List.select: I think that breaking up the select statement like this distorts and misrepresents the resource value and we should just print it in normal select statement format like we do in the present behavior, we should just not break it up into multiple lines if it is longer than 78 characters if this new option is specified.

2. Same comment as 3) except pertaining to schedselect attribute.

3. Similarly, I believe that we should leave Variable\_List as all one line as it is today, but simply not wrap it if it exceeds 78 characters.

4. The actual syntax used needs to be “qstat -f1” , rather than “qstat -f 1” (no space vs. space). The former can mean “full output with one line per attribute”, but the later already means “full output with line breaks for job 1.servername”.

---

<div class="post-metadata">

**Author:** ![nithinj](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/nithinj/32/45_2.png) [@nithinj](https://community.openpbs.org/u/nithinj)\
**Post date:** [November 14, 2016, 4:06pm UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/3 "2016-11-14T16:06:38Z")

</div>

Thanks Scott for the update.

I understand that this EDD is redundant in nature. So you can close the ticket-PP-508.  
You can ignore this EDD until there is a new update.  
Sorry for the inconvenience.

---

<div class="post-metadata">

**Author:** ![mkaro](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/mkaro/32/85_2.png) [@mkaro](https://community.openpbs.org/u/mkaro)\
**Post date:** [November 14, 2016, 7:07pm UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/4 "2016-11-14T19:07:54Z")

</div>

IMHO, PP-484 is the only one that matters for now. We might consider implementing the others in the future. We may also find that the others are unnecessary once PP-484 is complete.

---

<div class="post-metadata">

**Author:** ![nithinj](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/nithinj/32/45_2.png) [@nithinj](https://community.openpbs.org/u/nithinj)\
**Post date:** [November 17, 2016, 6:35am UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/5 "2016-11-17T06:35:19Z")

</div>

I’ve updated the EDD based on the comments received. Please review.

---

<div class="post-metadata">

**Author:** ![scc](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/scc/32/234_2.png) [@scc](https://community.openpbs.org/u/scc)\
**Post date:** [November 17, 2016, 4:06pm UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/6 "2016-11-17T16:06:00Z")

</div>

Hello Nithin, thanks for the update. There is one problem I see with the json example though. The “{” here is incorrect:

```
 "pbs_server":"vbox", 
 { 
      "Jobs"

```

And once that is removed a “}” needs to be removed from the end.

FYI, this is a handy tool when mocking up json: [http://jsonlint.com/](http://jsonlint.com/)

---

<div class="post-metadata">

**Author:** ![billnitzberg](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/billnitzberg/32/6_2.png) [@billnitzberg](https://community.openpbs.org/u/billnitzberg)\
**Post date:** [November 17, 2016, 4:31pm UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/7 "2016-11-17T16:31:59Z")

</div>

Hi,

I wanted to ask about the design for the option arguments (-Fjson and -Fdsv), and make sure the design follows standard option argument practice, as it’s not explicit in the design. Since the option arguments do not use “–”, I’m assuming the option arguments are not “Fjson” and “Fdsv”, but “F” alone with a required optarg.

Just want to verify that it is OK to have a space after the -F and before the optarg, e.g., the following are the same: “qstat -Fjson …” and “qstat -F json …”.

Thanks!

---

<div class="post-metadata">

**Author:** ![scc](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/scc/32/234_2.png) [@scc](https://community.openpbs.org/u/scc)\
**Post date:** [November 17, 2016, 5:48pm UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/8 "2016-11-17T17:48:06Z")

</div>

Hi Bill, good question, and yes, things like this work with pbsnodes today, so the expectation would be that qstat handles the arguments in the same way:

pbsnodes -av -F json

pbsnodes -avF json

pbsnodes -av -F dsv -D ,

pbsnodes -avF dsv -D ,

---

<div class="post-metadata">

**Author:** ![nithinj](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/nithinj/32/45_2.png) [@nithinj](https://community.openpbs.org/u/nithinj)\
**Post date:** [November 18, 2016, 5:12am UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/9 "2016-11-18T05:12:03Z")

</div>

Thanks, Bill and Scott.

I’ve updated the EDD with the following -

1. Removed JSON object opening before “jobs” and the corresponding closing tag.
2. Introduced a space between -F and optarg.

---

<div class="post-metadata">

**Author:** ![scc](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/scc/32/234_2.png) [@scc](https://community.openpbs.org/u/scc)\
**Post date:** [November 23, 2016, 9:18pm UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/10 "2016-11-23T21:18:28Z")

</div>

Looks good to me, thanks! I sign off.

Thanks,  
Scott Campbell

---

<div class="post-metadata">

**Author:** ![bremanandjk](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bremanandjk/32/144_2.png) [@bremanandjk](https://community.openpbs.org/u/bremanandjk)\
**Post date:** [November 24, 2016, 4:31am UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/11 "2016-11-24T04:31:01Z")

</div>

Looks good to me , You have my sign-off too.

Thanks,  
Brem

---

<div class="post-metadata">

**Author:** ![nithinj](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/nithinj/32/45_2.png) [@nithinj](https://community.openpbs.org/u/nithinj)\
**Post date:** [February 7, 2017, 6:32am UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/12 "2017-02-07T06:32:52Z")

</div>

I’ve updated the page with a “NOTE:” confirming that json and dsv will work only with the full set of attributes (while using “-f” option). Please share your thoughts.

---

<div class="post-metadata">

**Author:** ![vishwaks](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/vishwaks/32/74_2.png) [@vishwaks](https://community.openpbs.org/u/vishwaks)\
**Post date:** [February 7, 2017, 11:11am UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/13 "2017-02-07T11:11:13Z")

</div>

Hi Nithin,

EDD looks good. Below are few comments:

1. on -F dsv - What happens if a user give delimiter ‘,’ and an value of attribute contains it ? It will be escaped ?
2. on -F json - its better to provide all values in string format and requester will caste it accordingly. e.g., timestamp, Priority etc.

---

<div class="post-metadata">

**Author:** ![vishwaks](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/vishwaks/32/74_2.png) [@vishwaks](https://community.openpbs.org/u/vishwaks)\
**Post date:** [February 7, 2017, 11:36am UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/14 "2017-02-07T11:36:32Z")

</div>

And another thought. Its better to have it(json and dsv output) for below options also:

qstat -Bf - for server information  
qstat -Qf - for Queue information  
pbs\_rstat -f for Reservation information

Thanks,

---

<div class="post-metadata">

**Author:** ![nithinj](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/nithinj/32/45_2.png) [@nithinj](https://community.openpbs.org/u/nithinj)\
**Post date:** [February 7, 2017, 11:43am UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/15 "2017-02-07T11:43:35Z")

</div>

> [@vishwaks](#):
>
> on -F dsv - What happens if a user give delimiter ‘,’ and an value of attribute contains it ? It will be escaped

Good suggestion. We can print with escape character if the delimiter itself is appearing on the value.[quote=“vishwaks, post:13, topic:344”]  
on -F json - its better to provide all values in string format and requester will caste it accordingly. e.g., timestamp, Priority etc.  
[/quote]

The pbsnodes output in JSON format considers numeric values as a number ([http://www.json.org/](http://www.json.org/)). The same is followed in qstat.

---

<div class="post-metadata">

**Author:** ![nithinj](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/nithinj/32/45_2.png) [@nithinj](https://community.openpbs.org/u/nithinj)\
**Post date:** [February 7, 2017, 11:46am UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/16 "2017-02-07T11:46:47Z")

</div>

> [@vishwaks](#):
>
> qstat -Bf - for server information  
> qstat -Qf - for Queue information

This will be handled in the present enhancement. All the options of qstat with “-f”.

---

<div class="post-metadata">

**Author:** ![vishwaks](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/vishwaks/32/74_2.png) [@vishwaks](https://community.openpbs.org/u/vishwaks)\
**Post date:** [February 7, 2017, 12:18pm UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/17 "2017-02-07T12:18:18Z")

</div>

Good to know buddy. This will be helpful for PTL as well 👍

---

<div class="post-metadata">

**Author:** ![prakashcv13](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/prakashcv13/32/65_2.png) [@prakashcv13](https://community.openpbs.org/u/prakashcv13)\
**Post date:** [February 7, 2017, 3:48pm UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/18 "2017-02-07T15:48:38Z")

</div>

On the EDD page, could you please add a link to the community discussion.  
Also, for the “conflicting errors”, would you be adding a new message or an existing one will be reused?

---

<div class="post-metadata">

**Author:** ![nithinj](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/nithinj/32/45_2.png) [@nithinj](https://community.openpbs.org/u/nithinj)\
**Post date:** [February 8, 2017, 10:03am UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/19 "2017-02-08T10:03:25Z")

</div>

> [@prakashcv13](#):
>
> On the EDD page, could you please add a link to the community discussion.Also, for the “conflicting errors”, would you be adding a new message or an existing one will be reused?

`qstat: conflicting options.` is an existing error message. I’ve updated the EDD to clarify this. Also provided a link to community discussion.  
Thanks.

---

<div class="post-metadata">

**Author:** ![jon](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/jon/32/23_2.png) [@jon](https://community.openpbs.org/u/jon)\
**Post date:** [February 8, 2017, 3:44pm UTC](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344/20 "2017-02-08T15:44:32Z")

</div>

Sorry I have not replied sooner. I am not used to looking for EDD in user/site area. I would expect them to be in the developer area. Anyways, one concern that I have with doing json with qstat is that user environment variables can have characters that can break the json output. Are you doing anything special to ensure that this does not happen? If so, what?

[Next page](https://community.openpbs.org/t/pp-484-pp-485-enhance-output-of-qstat-to-make-it-more-admin-and-script-friendly/344.md?page=2)