Product Usage - Monsanto-NQS

Abstract

This paper looks at the results of an informal survey carried out amongst the UK Higher Education NQS user community six months after JISC NTI/48.2 began.

Contents

• Introduction
• Results
• Summary

Introduction

In response, an informal survey was conducted, sent to respondants to the original NQS questionaire, subscribers to our electronic mailing lists, and to users who had actually downloaded the NQS software itself.

No less than 29 replies were received within a week of the survey questionaire being sent out.

The Questionaire

> Hi, 
> 
> I'm currently the project officer for the support of public domain 
> NQS in the UK, and I'm conducting an informal survey to see whether 
> anyone is actually any of the services we're providing.
>
>  I see from our ftp logs that you have downloaded a copy of our 
> software, and have subscribed yourself to our NQS mailing list(s) 
> via Mailbase.
>
>  I would be most grateful if you could take a moment to answer these 
>  few questions.  If you are using NQS, or intend to use NQS, then 
>  please reply; we are currently bidding for extra funding, so that 
>  we can continue to support NQS, and if we cannot show that anyone 
>  is using NQS, then we will not get any extra funding, and NQS will 
>  be unsupported once more from the end of September this year.
>
>  o  Have you successfully downloaded our NQS product?
>
>     If so :
>
>     o  Have you installed/do intend to install NQS?
>     o  Do you intend to actually use NQS?
>
>     If not:
>
>     o  Do you intend to download our NQS product?
>     o  Would you be interested in our staff helping you install our
>        product?
>     o  Were you aware that we offer help for installing our NQS
>        product?
>
>  o  Do you consider our Mailing lists to be of any value at all?
>
>     If not :
>
>     o  Do you wish us to remove you from the mailing lists?
>
>  o  Do you consider our World-Wide Web service to be of any value 
>     at all?
>
>  o  Do you think that this JISC New Technologies Initiative project, 
>     for the support of UNIX batch systems, is of value to you?
>
>  o  Do you have any other comments?
>
>  I look forward to your reply, and many thanks.
>	
>  Stu

>  -----------------------------------------------------------------
>  Site			| Respondant
>  -----------------------------------------------------------------
>  Aston.1		| John Williams		WilliamJ@alpha.aston
>  Birmingham.1		| John Newbury		J.P.Newbury@bham
>  Brookes.1		| Colman Frank		p0060127@brookes
>  Cardiff.1		| Alan Oborne 		Oborne@taff.cf
>  Cardiff.2		| Andrew Cormack	scoanc@thor.cf
>  Cranfield.1		| Peter Lister		P.Lister@cranfield
>  Cranfield.2		| Mahan Janjua		mahan@rmcs.cranfield
>  Edinburgh.1		| Paul Haldane		Paul.Haldane@ed
>  Essex.1		| Roy Weaver		roy@essex
>  Imperial.1		| Ata Etemadi		atae@spva.ph.ic
>  King.1 (London)	| Andrew Beavil		ajb@helios.rai.kcl
>  Leeds.1		| Tim Nicholson		phy6tmn@irc.leeds
>  Leeds.2		| Jasbinder Singh	ecl6jasb@cif.leeds
>  Leicester.1		| D.G. Shores		dgs1@leicester
>  MCC.1		| Owen LeBlanc		LeBlanc@mcc
>  MCC.2		| M Whidby		M.Whidby@mcc.ac.uk
>  MCC.3		| Dave Haworth		zzassdh@cs6400.mcc
>  Oxford.1		| Malcolm Beattie	mbeattie@sable.ox
>  Queens-Belfast.1	| John Pelan		J.Pelan@Queens-Belfast
>  Rutherford.1		| Ivan Fabian		igbf@triumph.cc.rl
>  Salford.1		| Alan Goodkin		A.R.Goodkin@ais.salford
>  Sheffield.1		| Chris Cartledge 	C.Cartledge@sheffield
>  Sheffield.2		| Mark Taylor		M.Taylor@sheffield
>  Sheffield.3		| Linday Gray		L.Gray@sheffield
>  Sheffield.4		| Ian Staniforth	I.Staniforth@sheffield
>  Stirling.1		| Brian Bullen		B.M.Bullen@stir
>  Strathclyde.1	| Jim Gentles		craa24@ccsun.strath
>  Ulster.1		| R Wilson		R.Wilson@ulst
>  Ulster.2		| Stephen McFarland	SAH.McFarland@ulst
>  -----------------------------------------------------------------
>  Total		| 29 replies
>  -----------------------------------------------------------------

>  -----------------------------------------------------------------
>  Site			| Download	| Install/	| Use/
>			|		| Intend to	| Intend to
>  -----------------------------------------------------------------
>  Aston.1		| Yes		| Yes		| Yes
>  Birmingham.1		| No		| Yes		| Yes
>  Brookes.1		| Yes		| Yes		| Yes
>  Cardiff.1		| Yes		| Yes		| Yes
>  Cardiff.2		| Yes		| Yes		| Yes
>  Cranfield.1		| Yes		| Yes		| Yes
>  Cranfield.2		| Yes		| Yes		| Yes
>  Edinburgh.1		| Yes		| Probably	| Maybe
>  Essex.1		| Yes		|		|
>  Imperial.1		| Yes		| No		| No
>  King.1		| Yes		| Yes		| Yes
>  Leeds.1		| Yes		| Yes		| Yes
>  Leeds.2		| No		| Perhaps	| Perhaps
>  Leicester.1		| No		| Yes		| Yes
>  MCC.1		| No		| No		| No
>  MCC.2		| No		| No		| No
>  MCC.3		| No		| No		| No
>  Oxford.1		| Yes		| Yes		| Yes
>  Queens-Belfast.1	| Yes		| Yes		| Yes
>  Rutherford.1		| No		| Yes		| Unsure
>  Salford.1		| No		| Yes		| Yes
>  Sheffield.1		| Yes		| Yes		| Yes
>  Sheffield.2		| Yes		| Yes		| Yes
>  Sheffield.3		| Yes		| Yes		| Yes
>  Sheffield.4		| Yes		| Yes		| Unsure
>  Stirling.1		| Yes		| Yes		| Yes
>  Strathclyde.1	| Yes		| Yes		| Possibly
>  Ulster.1		| Yes		| Possibly	| Possibly
>  Ulster.2		| Yes		| Yes		| Possibly
>  -----------------------------------------------------------------
>  Total	| 29	| Y : 21	| Y : 21	| Y : 17
>  -----------------------------------------------------------------

• Brookes are using Monsanto-NQS on Sun machines, linking with Sequent's NQS on Sequent machines.

• Leeds.1 deleted NQS, but intend to reinstall on a new server machine in the near future.

• Leeds.2 currently run 4D/NQS, and will look for an alternative only when 4D/NQS no longer works for them.

• MCC have their own, in-house version of NQS.

• Rutherford are desperate for proper scheduling, and would consider switching from CERN-NQS if this was implemented.

• Sheffield.4 intend to install so that they can evaluate the product themselves.

>  -----------------------------------------------------------------
>  Site			| Install	| WWW		| Mailbase
>  -----------------------------------------------------------------
>  Aston.1		| Yes		| No		| No
>  Birmingham.1		| Vaguely	| Yes		| Yes
>  Brookes		|		| Yes		| Yes
>  Cardiff.1		|		| Yes		| Yes
>  Cardiff.2		|	 	| Yes		| Yes
>  Cranfield.1		| Yes		| Yes		| Yes
>  Cranfield.2		| 		| 		| Yes
>  Edinburgh.1		| Yes		| Yes		| Yes
>  Essex.1		|		|		|
>  Imperial.1		|		| Yes		| Yes
>  King.1		| 		| No		| Yes
>  Leeds.1		|		| No		| No
>  Leeds.2		|		|		|
>  Leicester.1		| Yes		| Yes		| Yes
>  MCC.1		| Yes		| Yes		| Yes
>  MCC.2		| No		| Yes		| Yes
>  MCC.3		| Yes		| Yes		| Yes
>  Oxford.1		|		|		|
>  Queens-Belfast.1	| Yes		| Yes		| Yes
>  Rutherford.1		| No		| Yes		| Yes
>  Salford.1		| No		| Yes		| Yes
>  Sheffield.1		| Yes		| Yes		| Yes
>  Sheffield.2		| Yes		| Yes		| Yes
>  Sheffield.3		| Yes		| Yes		| Yes
>  Sheffield.4		| 		| 		| No
>  Stirling.1		| Yes		| Yes		| Yes
>  Strathclyde.1	| Yes		| Yes		| Yes
>  Ulster.1		|		|		| Yes
>  Ulster.2		|		| No		| No
>  -----------------------------------------------------------------
>  Total	| 29	| Y : 12	| Y : 19	| Y : 22
>  -----------------------------------------------------------------

>  -----------------------------------------------------------------
>  Site			| WWW		| Mailbase	| Project
>  -----------------------------------------------------------------
>  Aston.1		|		|		| Yes
>  Birmingham		| Possibly	| Yes		| Yes
>  Brookes.1		| Yes		| Yes		| Yes
>  Cardiff.1		| Yes		| 		| Yes
>  Cardiff.2		| Yes		| Yes		| Yes
>  Cranfield.1		| Yes		| Yes		| Yes
>  Cranfield.2		| 		| Yes		| Yes
>  Edinburgh.1		| Yes		| Yes		| Yes
>  Essex.1		|		|		|
>  Imperial.1		| Yes		| No		| No
>  King.1		|		| Yes		| Yes
>  Leeds.1		|		|		| Yes
>  Leeds.2		|		|		|
>  Leicester.1		| Yes		| Yes		| Yes
>  MCC.1		| Yes		| Yes		| Yes
>  MCC.2		| Yes		| Yes		| Yes
>  MCC.3		| No		| No		| No
>  Oxford.1		|		|		|
>  Queens-Belfast.1	| Yes		| Yes		| Yes
>  Rutherford		| Yes		| Yes		| Yes
>  Salford.1		| 		| Yes		| Yes
>  Sheffield.1		| Yes		| Yes		| Yes
>  Sheffield.2		| Yes		| Yes		| Yes
>  Sheffield.3		| Yes		| Yes		| Yes
>  Sheffield.4		|		|		| Possibly
>  Stirling.1		| Yes		| Yes		| Yes
>  Strathclyde.1	| Yes		| Yes		| Yes
>  Ulster.1		| 		| Yes		| Yes
>  Ulster.2		| Yes		| Yes		| Yes
>  -----------------------------------------------------------------
>  Total	| 29	| Y : 17	| Y : 20	| Y : 23
>  -----------------------------------------------------------------

• I think this is an excellent project where the output are directly helping UK HE make more effective use of equipment through the utilisation of conventional technologies, batch systems, aout which there has been insufficient information or support for in the past.

• It has been valuable for us to have contact with and to interchange ideas with other sites using NQS.

• No, we haven't downloaded it yet, but intend looking at Monsanto-NQS as a replacement for our 4D NQS system this summer. We are aware of the installation service but would want to carry this out ourselves. However, some kind of support resource will be useful in my opinion.

  I have to admit that I haven't seen a lot from the mailing lists. I think I'm only on the announce list, so that might explain it. I think that they could provide a very useful medium for communicating info about the NQS product and its support.

  Any support for products widely used within the University sector is welcomed as it saves a lot of hassle in solving every little bug yourself.

• We are currently using version 3.35, downloaded from Monsanto last year, to provide a reasonably popular batch service. I have downloaded and compiled 3.36.6 from you, but was diverted onto other things before I could fully test or release it. As far as I know we intend to continue using NQS for the foreseeable future.

  (re Mailbase lists - Stu) Yes, very much so. I obtained a lot of help in getting NQS compiled from the list.

  (re Project being of value - Stu) Yes. It provides a responsive support line.

• It is installed and is in use daily on a small network of SGI Indigo's where it has proved to be very useful in balancing the use of the machines. I previously used a copy of CERN NQS, but I have found Monsanto-NQS to be far superior.

  The mailing lists has been useful to me mainly for flagging new versions and bugs.

  The project is very definitely of value to us. Please keep at it, I'm very encouraged by the advances made to date.

• I have not got as far as offering a user service. Partly this is because my time is extremely limited, and the NQS product appears to require a certain amount of setting up, for which I have not had time so far.

  It would be useful to have some typical configurations ready to run, eg

  config A -

  1 cluster of 10 unix workstations to run with loadbalancing job submissions from any of this 10

  config B -

  config A, plus job submissions from a number of other unix hosts, which would not actually run any tasks

  config C -

  2 set of unix workstations, 5 to run long running tasks, 5 to run tasks < 5 mins

  config D -

  mixture of A, B and C

• No. I have not downloaded your NQS product yet. We currently use CERN-NQS and it is not clear the Sheffield/Monsanto-NQS has anything additional to offer us in its current state. We are desperate for the scheduling facilities you have down in the work list.

  Just for interest: If we did decide your NQS product I would see it being used on quite a few machines. To start with we may run it on 6 DEC Alpha machines, and 20 HPs.

  (Re - intend to download our NQS product - Stu) I do, but am waiting for it to progress a bit further. If it would be of any value to you I could download it and give it a bit of a trial on my desktop DEC Alpha machine but would not be able to spend very much time testing/trying it out.

  I did not have problems installing CERN-NQS so hopefully I'd be able to install your offering. I may appreciate a help line of sorts in case I ran into problems.

  (Re mailing lists being of value - Stu) Yes. I want to be kept informed of your progress.

  (Re WWW service being of value - Stu) Yes. I have pointed several people at it to give them an idea of the project and the current situation.

  I would like to see this project continue. We make a lot of use of CERN-NQS already but are unhappy about the current scheduling capabilities.

• (Re project being of value - Stu) Yes - a focus for work on a particular product which is of obvious interest to the UK academic community has to be a good thing.

  We have a specific interest in comparisons between batch systems like NQS and systems like LSF and NClogin/NCshare.

• If NQS was much simpler to install and required less changes and edits to various system files I would certainly have considered installing it. It would also be more useful if there were other nodes outside my cluster which could take on some of the work load. Otherwise using the unix batch facility is much simpler and NQS offers no additional advantages.

• (1) The project should be more concerned with development of a solid, portable NQS rather than support. A well prepared package with good documentation needs little support if any. It could be argued that anyone needing support shouldn't be installing it.

  (2) I don't agree with your directions into C++ on the grounds of portability and maintainence. Take a look at ftp://ftp.parc.xerox.com/pub/ilu/ilu.html.

  (3) A GUI for administration and monitoring is beneficial but should not be a priority. Motif and/or Athena should beused or failing that use Tk/Tcl. Stay away from proprietary systems.

  (4) Proposing an open standard for a distributed computing system is a Good Thing, which you are doing. Keep track of any OSF developments. Get in contact with GNU ensure that they know what is going on. Just because you are JISC funded doesn't mean you can't get help.

  (5) Get in contact with the Distributed Computing Support Programme based at the Daresbury Laboratorry. Support is their job. See http://www.dl.ac.uk/TCSC/disco/main.html.

  (6) I suspect your original poll was sent to computer centres. This was a mistake. In my experience computer centre personnel have little experience with real world applications and didn't know what to do when you mentioned NQS. Of course many more people will be using NQS than a computer centre person would be aware of. Needless to say our CC never contacted anyone else.

  (7) Few distributed queueing systems address the issue of running batch jobs over NFS links - this is a BAD THING (poor through-put, big network traffic and very inefficient). This should be addressed. Similarly not having accounts on the remote machines would be a useful feature.

  (8) How about logging all the NQS mailing lists on the NQS WWW site using, say, MHonArc or Hypermail. That way people can delete old mail knowing they can always check the archive easily via WWW.

• Although I intend to install NQS, I may need to call on you for information and advice.

• I shall wait until I have my server to play with. It must be better than `at' queues. In particular Biosym software have a feature to direct background jobs to NQS queues which could be of use.

• (Re do actually using NQS - Stu) Almost certainly. We installed DQS, which appeared to suit us best at the time, but has now develped into a different and incompatible system. We now also have a Cray running NQS, and we would like to integrate with that and any other UK systems.

  (Re project being of value - Stu) Potentially of great value. I have had other priorities recently, but I would like to see a decent quality widely used batch queueing system. It would be of great benefit to our users.

  Some local wishes are

  Co-existence with existing authentication systems (eg Kerberos, AFS, DCE). Compatability amongst workstations, our Cray and potentially off-site, eg national centres. This effectively means a well designed batch queueing protocol. Support fo PVM/MPI.

• What is needed for all these things is some kind of central reference point so that contacts, servers htmls etc can be found easily.

• We are not actively looking at an alternative to 4D/NQS. However, if we have problems running 4D/NQS at IRIX 5.2/5.3, then we will have to look at an alternative. We have Sterling NQS which we have not used so far but this may be what we use if we cannot continue with 4D/NQS. Alternatively, we may look at installing the public domain version ie Monsanto NQS.

• The main reason at the moment for installation was the ACL controlled device queues although we may well use batch queues soon. I had to uninstall the whole package soon after the installation when I discovered awful security holes in most of the executables. Anyone writing suid executables should not be using unchecked strcpy's and strcat's all over the place. I managed to put in kludges to plug up the worst of the instances but I'm not convinced that the rest of the code is secure.

• (Re project being of value - Stu) I guess the service will be valuable if we go ahead.

• The above mentioned information services (WWW, Mailbase - Stu) would be useful for support.

• Apologies for being so negative, but since we were so involved and established with our own version of nqs, I guess my replies are not unexpected.

• We currently have an old NQS which we know works on our hardware, but we are aware of its limitations. There is nothing more frustrating than unsupported software. We have a slightly unusual configuration, but do not have the resources to tinker with software ourselves.

  As we are likely to be upgrading our compute service fairly soon, support of the same batch system on different hardware could be a positive advantage.

• If you install the software in a directory then you cannot install it in different unless you start from scratch again ie compiling etc.

• (Re the project being of value - Stu) Yes - without this initiative, we may never have thought of implementing a batch system on our large Unix Servers.

  A course on NQS, it's advantages & disadvantages and an indication of the type of environment it would work well & or badly in (would be useful - Stu).

• 64 individuals were contacted via electronic mail. We received 29 replies (45.3%)

• 21 respondants have downloaded the NQS source code (72.4%).

• 21 respondants have installed NQS, or intend to install NQS (72.4%). A further 3 respondants are considering installing NQS for evaluation purposes (10.3%).

• 17 respondants intend to actually use NQS (58.6%). A further 7 respondants are undecided at this time whether they will actually use NQS or not (24.1%).

• 12 respondants were aware of the installation service offered by this project (41.4%). A further 13 respondants did not indicate either way (44.8%).

• 19 respondants were aware of the World-Wide Web service offered by this project (65.5%). A further 6 respondants did not indicate either way (20.7%).

• 22 respondants were aware of the Mailbase service offered by this project (75.9%). A further 3 respondants did not indicate either way (10.3%).

• 17 respondants said that the World-Wide Web service provided by this project was of value to them (58.6%). A further 10 respondants did not indicate either way (34.5%).

• 20 respondants said that the Mailbase electronic mailing lists provided by this project was of value to them (69.0%). A further 7 respondants did not indicate either way (24.1%).

• 23 respondants said that JISC NTI/48.2 was of value to them (79.3%). A further 4 respondants did not indicate either way (13.8%).