NASIG | North American Serials Interest Group

About NASIG

NASIG EVALUATION & ASSESSMENT COMMITTEE MANUAL

PDF Version
(see private E&A committee webspace for the editable word.doc version)

NASIG Information
NASIG bylaws
Working calendar

Committee General Information
Committee charge and current roster
NASIG committee guidelines

Procedures

The timing of several of the procedures is related to the dates of the conference, therefore an exact timeline cannot be prescribed here. The committee may also be asked to provide evaluation-related services for continuing education events or ad hoc surveys for other committees. Because the timing of these events may vary, they are not included in the timeline.

PLEASE NOTE: All announcements sent externally (via SERIALST, e.g.) need to get prior approval from the NASIG publicist.

Monthly

As appropriate, submit updates and announcements to the Publicist for the "What's New" section of the web site.

January

The Committee’s membership runs on the calendar year, so the new chair will begin his or her term January 1.

The chair should consult with the Board Liaison to verify the committee membership roster for the year.

Because most committees change over in July, it is a good idea for the chair to contact the Electronic Communications Committee to remind them about the E&A Committee changes and to verify that committee lists are updated for the committee discussion list and on the Web site. Also verify that the committee e-mail address evaluation@nasig.org is being routed to the new chair.

Test the committee discussion list. Remind committee members that the majority of the committee's work occurs the month prior to the conference and the month (or two) following the conference. Committee members should be available to complete time-sensitive tasks during those months.

At this time the new chair needs to make some appointments:

  • The Committee chair selects and appoints a committee member to serve as the Web liaison. Each committee should have a Web liaison. Ideally, the chair will ask for a volunteer and someone on the committee will respond. If no one volunteers, the chair will appoint someone. NASIG will try to host a brief training session during each annual conference for all committee web liaisons. If at all possible, the Web liaison should try to attend this training session, though it is not required. The duties of the Web liaison are:
    • Load documents that need to be shared among the committee members so that committee's shared spaces on the NASIG website and any other spaces such as GoogleDocs, etc.
  • Communicate to the ECC any changes that need to be made to the committee's Web space and elsewhere on the NASIG Web site regarding that committee. The Committee chair also selects and appoints a Committee member for the calendar year to monitor the Discussion Forums and help drive discussion topics to keep members informed and connected.
  • The Committee chair should identify a Committee member to serve as vice chair/chair-elect for the year.

Notify the Board liaison of all of these appointments.

 

The Executive Board generally meets in January. The chair from the previous year should have prepared and submitted an annual report for the previous calendar year and submitted it to the Board Liaison. At this January Board meeting, the Board liaison will request input on any desired revisions to the conference evaluation process and communicate any back to the Committee.

February/March

The president-elect makes committee assignments for the coming year. Because E&A operates on a calendar-year cycle, new members appointed at this time do not begin their terms until the following January.

The chair should communicate with the Program Planning Committee and Conference Planning Committee chairs about the evaluation survey for the upcoming conference. Find out an approximate date for release of the conference program so you can plan for the committee to build the evaluation survey. Also, let the PPC and CPC chairs know that they are free to request that specific questions be added to the evaluation survey.

The committee may also wish to review the standard questions and answers from the previous year's evaluation survey to determine if any changes should be made.

Contact the Executive Board about any supplemental questions they would like added to the evaluation survey. Find out if there will be a drawing for a free registration for those individuals filling out the evaluation survey.

Month Prior to Conference - April or May

Once the conference program is published, you can build the evaluation survey on SurveyMonkey.

Review the full program and conference Web site to look for any new features/events to add questions about to evaluation survey.

The SurveyMonkey user name and password can be obtained from the ECC. A Statement of Confidentiality form should be signed by anyone who receives the user name and password and submitted to the Archivist.

Some helpful tips for creating the survey:

  • When creating a new survey, you can copy an existing survey. This will save you from having to retype the standard questions.
  • Use the "Rating Scale" question type for any question you want a numeric rating computed for (i.e., all program and presenter ratings).
  • Use question logic to direct respondents to (or allow them to skip) questions for pre-conferences and poster sessions.
  • The "Survey Options" button allows you to add the NASIG logo to the survey.
  • The "Collect Responses" tab has a number of settings you should review. This is also where you get the url for the survey to post on the conference Web site.
  • If there is a drawing for a free registration, when the evaluation survey is finished you can redirect respondents to a separate survey to enter their names into the drawing. This will insure that survey responses remain anonymous. At the end of the drawing survey, redirect respondents to the NASIG Web site. The redirect option is found in the "Collect Responses" section.
  • Once the survey has been created and the committee has reviewed it, be sure to clear all responses before the evaluation survey goes live. Expect changes to the program to continue until the conference starts. Most of these changes will require adjustments to the survey.

 

Establish the deadline for the evaluation survey. Generally leave the survey open for 3-4 weeks. Traditionally the deadline has fallen on July 1st ( specify the time of the deadline, i.e. 7 p.m. CST). Earlier or later conference dates will alter this.

One to two weeks before the conference, submit the survey url and related text for the evaluation page on the conference Web site to the Electronic Communications Committee &/or the conference web designer. Specify that the page should be updated and made available before the start of the conference. Double-check to make sure the link and text get posted.

Approximately one week prior to the conference, coordinate with publicist to send messages to members, and others as appropriate, reminding attendees that evaluations will be online with a PDF of the survey, including a link to the survey. The announcement should also include specific information about the survey's closing date & time (e.g., 11:59 PM 1 July). Use the ArcStone admin module to send out a blast message to registered conference attendees to let them know that the evaluation survey will be available online with a PDF of the survey, the url for the conference evaluation page, and the deadline. If there will be a drawing for a free registration for evaluation survey respondents, mention that as well. With permission of the publicist, you may also want to send a message via a more general listserv, e.g. SERIALST.

Sample message text:

The Evaluation & Assessment Committee will be collecting your feedback on the 20XX NASIG Annual Conference in CITY, STATE. Evaluation survey will be available online only. Conference attendees who complete the online evaluation survey will be eligible to enter a drawing to win a FREE registration to the 20XX or 20XX NASIG Annual Conference.

The link to the online evaluation survey is posted on the conference Web site: http://www.nasig.org/conference_evaluation.cfm

As in previous years, you will be asked to rate the sessions you attended using a five-point scale, with 5 being the highest rating and 1 being the lowest. You will be asked to rate each presenter, as well as provide an overall rating for the session. You will also have the opportunity to rate other conference events, facilities, and arrangements and to provide input for next year's conference. The deadline for filling out the evaluations is HOUR:MINUTE AM/PM MONTH DAY, YEAR.

NOTE: All questions or problems encountered with the NASIG (ArcStone) website should be routed to the ArcStone liaison - Buddy Pennington.

Also remind the CPC chairs, President, and President-Elect that the committee would appreciate it if they would remind assembled conference attendees about the evaluation survey during the conference.

A brief committee report is due for the Executive Board meeting held during the conference. The committee can use the report to submit recommendations or actions items to the Executive Board. The Board Liaison typically sends a reminder in advance of the meeting.

Conference and Post-Conference - May or June

The committee meets during the annual conference. New members appointed in the spring are encouraged to attend the committee meeting, but they should not be added to the committee discussion list or included in committee conversations about evaluation results until their appointments officially begin the following January.

There is usually a chairs orientation/meeting during the conference. Both the current chair and vice-chair may attend. Both may also wish to attend training that ECC often offers in working with the ArcStone software that provides the architecture and functionality for the NASIG web site.

Following the conference, use the blast messaging system to send out reminders to conference attendees about filling out the evaluation survey. You will want to send out periodic reminders about the evaluation survey throughout the period the survey is open. With permission from the Publicist, you can also use serials-related electronic discussion lists for reminders, for example SERIALST.

Once the survey deadline passes, be sure to close the collector in SurveyMonkey so the survey is no longer collecting responses.

Other NASIG Committees will have their memberships changing over following the conference. Remind the Electronic Communications Committee that E&A membership will not be changing and the incoming members for the next calendar year cannot be added to the committee discussion list at this time due to the confidential nature of the committee's work. You may optionally wish to re-test the committee list to make sure it is only reaching current committee members. Alternately, when sending out confidential information, you may choose to by-pass the committee discussion list and send e-mails directly to each individual committee member.

Conference Drawing - June or July

Once the evaluation survey is closed, if the Board has approved a drawing for a free registration, follow these procedures:

  • Download the data from the drawing survey into Excel (or other spreadsheet software) and de-duplicate so there is only one entry per person.
  • Excel (or other spreadsheet software) should automatically number the responses.
  • A committee member other than the chair (or whoever generated the list of drawing entrants) will be given the number of people in the drawing and go to www.random.org (a random number generator) to select the number of the winner. For example, if there are 300 people in the drawing, have the random number generator generate a number between 1 and 300. The number generated will be used to select the winner from the numbered list of entrants.
  • Notify the winner, copying the NASIG President and the E&A Committee co-chair on the e-mail. After the winner has accepted the prize, submit a brief write-up to the NASIG Newsletter editor for publication in the blog.

 

Text for the message to the drawing winner:

It is my privilege to inform you that you are the winner of the drawing for a free NASIG conference registration! Please review the terms below and confirm your acceptance of them to me by e-mail.

  • The conference registration may be used for either the 20XX or 20XX conference.
  • The conference registration may be transferred to another NASIG member.
  • To redeem your free registration, please contact the conference registrar prior to registering. The registrar will provide you with instructions for filling out the registration form so your conference fee will be waived.
  • This prize includes basic conference registration only; pre-conferences, tours, social events, hotel accommodations, and travel expenses are not included.

 

Thank you so much for taking the time to fill out the online evaluation survey.

Evaluation Survey Wrap-Up - June/July or July/August

Once the collector is closed in SurveyMonkey, use the "Analyze" button for your survey to obtain the results. On the left-hand side of the screen, there should be a button for "Share Responses." Select "Share Summary of Responses + View Open-Ended Responses" and the software will provide you with the url for the results page that you will share with the Executive Board.

Write up a summary report of the conference evaluation results. Examples of previous reports can be found in the NASIG Newsletter.

  • A full report (marked "Confidential") will be submitted to the Executive Board. This will contain the conference evaluation report, the url for raw survey results, and any confidential comments about the survey process you wish to pass on to the Board. This version of the report is also shared with the incoming chairs of the Program Planning Committee and Conference Planning Committee, and they should be reminded that it cannot be shared with PPC and CPC committee members. All recipients of this version of the report should be reminded that it is CONFIDENTIAL.
  • An edited version of the conference evaluation report only will be submitted to the NASIG Newsletter. This version of the report is also shared with the Conference Planning Committee and the Conference Coordinator. Raw survey results are considered CONFIDENTIAL and are NOT shared with the membership.

The Program Planning Committee will send out thank you letters to each conference presenter. Presenters will be told to contact the E&A Committee (via evaluation@nasig.org) if they wish to receive individual evaluations for their presentations.

  • One or more committee members can be designated to provide individual evaluation results.
  • Provide presenters with the individual rating, the overall rating, and any comments. Do not send individual ratings for any co-presenters (if there are any) and also scan the comments and remove anything specifically aimed at a co-presenter. Use your judgment as to what best respects the confidentiality of any co-presenter's evaluation comments.
  • Remember the results are highly confidential and the Committee may not share results for a speaker with anyone other than that individual, even if they presented together on the same program or panel.
  • Be mindful of sending these out in a timely manner. Presenters in faculty positions often need ratings for tenure, promotion, or reappointment cases.
  • Sample text for the letters send to presenters:

    [NAME]:

    Per your request, here are the evaluation results for the [program/pre-conference/poster session] "[NAME OF PROGRAM]," which you [presented/co-presented/moderated/served as a panelist for] at the 20XX Annual Conference of the North American Serials Interest Group (NASIG).

    Ratings are made on a 5-point scale, with 1 being the lowest rating and 5 being the highest rating. Percentages reflect the proportion of attendees selecting the rating, followed the actual number in parentheses:

    John Smith
    1: 0% (0)
    2: 0% (0)
    3: 28.3% (13)
    4: 45.7% (21)
    5: 26.1% (12)
    Rating average: 3.98
    Program Overall
    1: 0% (0)
    2: 0% (0)
    3: 29.5% (13)
    4: 40.9% (18)
    5: 29.5% (13)
    Rating average: 4.0
    Comments:
    1. Interesting presentation. Well done and informative.
    2. This topic is very relevant to my work. The detailed account of your methodology for analyzing statistics is especially appreciated etc.

 

Late Summer/Early Fall

Contact the chair of the Continuing Education Committee to remind him or her that the E&A Committee can provide evaluation-related services for any continuing education events they sponsor.

The NASIG Treasurer will contact you about the upcoming year's budget. With online-only surveys, the committee's expenses are generally low or non-existent (the survey software is paid for from a different budget). It is recommended that some money be put into the budget to cover unforeseen expenses or projects.

The Executive Board's fall meeting generally occurs in September or October. The committee will be asked to submit a brief report covering activities occurring since the annual conference. The committee can use the report to submit recommendations or actions items to the Executive Board.

November/December

The chair and vice-chair review the committee's procedure manual for any needed updates. If the manual requires any revisions, be sure to submit the revised version to the Board Liaison and the Electronic Communications Committee for posting on the committee's page on the NASIG Web site.

The chair prepares the committee's annual report to be submitted to the Executive Board. Examples can be found in the NASIG Newsletter. The committee can also use the report to submit recommendations or actions items to the Executive Board.

The chair sends out thank you notes to committee members.

Miscellaneous

Teleconferencing should be handled by Skype wherever possible (some institutions do not allow downloads on university/company/etc. computers).

Access to the board list email archives: http://nasigeccmanual.pbworks.com/Checking-Listserv-Archives

Rev. 11/2011



25th Anniversary Task Force
Archivist
Awards and Recognition
Bylaws
Committee Chairs
Committee Liaison
Conference Planning
Conference Proceedings Editors
Continuing Education
Core Competencies Task Force
Database and Directory
Electronic Communications
Evaluation and Assessment
Executive Board
Financial Development
Membership Development
Mentoring Group
Newsletter
Nominations and Elections
Program Planning
Publications and Public Relations
Site Selection
Student Outreach