Jimi,
You raise an interesting point about the response scales used. For the
social sciences these are generally based on the Likert Scale --
http://en.wikipedia.org/wiki/Likert_scale. Jimi's point about "forced
choice" is discussed here in the entry. I think there are varying
opinions on whether including a neutral category or making the item
forced choice matters -- the Wikipedia entry says the difference is
negligible.
I took a look at three of ICPSR's most frequently downloaded studies --
the National Survey of Adolescent Health, the General Social Survey, and
the American National Election Study. GSS uses forced choice (but
includes the "don't know" option) while the other two do not. We would
probably find lots of variation across the whole ICPSR collection. I am
thinking our survey is probably OK either way.
Mary
-----Original Message-----
From: The NDSA Standards working group list
[mailto:[log in to unmask]] On Behalf Of Jones,
Jimi
Sent: Tuesday, May 22, 2012 1:40 PM
To: [log in to unmask]
Subject: Re: [NDSA-STANDARDS] please review revised version of the
staffing survey
Hello all. I have some stray thoughts about the survey (which is looking
great, y'all).
First, in Q4 maybe we could say for profit "entity" instead of
"corporation." This might make it a bit broader for the for profit
folks.
Something else caught my eye. In Q15 we have a "Neither Agree nor
Disagree" answer option and in Q17 we have a "Neither Important nor
Unimportant" answer option. As I think back to my days as a Psych major
- one who had to put together a LOT of surveys - I remember being told
by one of my professors that a good survey does its best to avoid having
a "neutral" answer because people are, for whatever reason,
disproportionately drawn to checking the box in the middle. A good
survey leaves out this middle-of-the-road bias by having an even number
of choices without a "neither here nor there" option. Yes, this gently
tugs the user to make a decision ("well I don't have an 'OK' response
option so I have to really consider my choice"), which may seem
controversial to some folks but in my experience it really helps to
break apart the bell curve, which can be more diagnostic. So I would
recommend excising the "Neither Agree nor Disagree" and "Neither
Important nor Unimportant" answer options. Thoughts?
And did we decide on what to say about ourselves at the beginning of the
survey? I think we should say something like "This survey was put
together by the Standards and Best Practices Working Group of the
National Digital Stewardship Alliance (NDSA). For more about the Working
Group go here:
http://www.digitalpreservation.gov/ndsa/working_groups/standards.html
and for more about the NDSA, including how your institution can become a
member, go here: http://www.digitalpreservation.gov/ndsa/."
By the way, I'm getting the posts to this listserv in strangely
non-chronological order. My apologies if someone has already pointed
these things out. Glad to see the discussions on the list though!
Best,
Jimi
-----Original Message-----
From: The NDSA Standards working group list
[mailto:[log in to unmask]] On Behalf Of
Goethals, Andrea
Sent: Tuesday, May 22, 2012 12:18 PM
To: [log in to unmask]
Subject: Re: [NDSA-STANDARDS] please review revised version of the
staffing survey
Hi Amy,
> -----Original Message-----
> From: The NDSA Standards working group list [mailto:NDSA-
> [log in to unmask]] On Behalf Of Amy Kirchhoff
> Sent: Tuesday, May 22, 2012 12:10 PM
> To: [log in to unmask]
> Subject: Re: [NDSA-STANDARDS] please review revised version of the
> staffing survey
>
> Hi Andrea ~
>
> I like your proposal for Q10.
[AG: ] Great. Before making this change I'll see if we get more feedback
on it.
Speaking of big changes -- Mary had a good idea I want to run by you
all. The question that I believe was Q11 asking "How many people in your
organization perform digital preservation work either full or part time"
is related to the question (Q14) that asks number of individuals
performing specific dp functions. We could infer the overall number of
indivs. by adding up the numbers in Q14. We wouldn't get FTE though
(unless we changed #14). I noticed in the DigCurV survey they only asked
about FTE and not number of indivs. Any thoughts in the group about
getting rid of Q11 and/or asking about FTEs in Q14 instead of number of
indivs.?
>
> For Q16, I would just propose we change the text of the question to be
> "and/or" to match the selector choice. :-)
[AG: ] Gotcha!
Andrea
>
> ~ Amy
>
> -----Original Message-----
> From: The NDSA Standards working group list [mailto:NDSA-
> [log in to unmask]] On Behalf Of Goethals, Andrea
> Sent: Tuesday, May 22, 2012 11:53 AM
> To: [log in to unmask]
> Subject: Re: [NDSA-STANDARDS] please review revised version of the
> staffing survey
>
> Hi Amy,
>
> Yes, Q10 is complex. You are right that the 2nd to last column is
> intended to mean we're not currently doing it but would like to. We
> could make the column heading say explicitly that.
>
> We can also split this up into multiple questions as I started to with
> adding #9. Based on feedback from Mary I think that #9 should be
> reworded to "Which of these activities are considered part of the
> digital preservation function at your organization, whether or not you
> are currently doing/outsourcing this activity?" (Because there was
> ambiguity as to you if they should only select things they are
> currently doing/outsourcing). Is the proposed rewording clear enough?
>
> We're currently asking many things in #10:
> - whether or not they are currently doing them
> - if they are currently doing them whether its in-house, outsourced
> (or both because you can select more than one)
> - in the case that they are not doing it, if they would like to add it
> to their program (actually this may be redundant information. With the
> proposed rewording of #9 we will know that any activities listed in
> #10 that they didn't say they are currently doing are activities they
> would like add)
> - whether they would like to outsource activities
>
> Here's one option for #10. We could simplify it to just have the first
> 2 columns (currently done in-house, currently outsourced). Then we
> could have a follow-up question that lists the same activities as #10
> and asks which they would like to outsource (we would know that any
> they don't select they would want to do in-house).
>
> --------
> Q16 - Currently they can select more than one so checking both asks as
> an and. We could change it to be more explicit with radio buttons and
> have an option that includes both of the first 2 options. Does that
> work better?
>
> Andrea
>
>
>
>
>
>
> > -----Original Message-----
> > From: The NDSA Standards working group list [mailto:NDSA-
> > [log in to unmask]] On Behalf Of Amy Kirchhoff
> > Sent: Tuesday, May 22, 2012 11:12 AM
> > To: [log in to unmask]
> > Subject: Re: [NDSA-STANDARDS] please review revised version of the
> > staffing survey
> >
> > Hi all ~
> >
> > On Q10, I'm not quite sure what we mean by the 2nd to last column.
> >
> > Are we saying this is not currently done, but we would like to do
it?
> > And then we could click Would like to outsource, as well?
> >
> > So, one could combo:
> >
> > * Currently done in-house and would like to outsource.
> > * Would like to add to our digital preservation program and would
> like
> > to outsource.
> >
> > But the other combos really only mean you are currently doing two
> > things?
> >
> >
> > I guess, I'm stumbling because it seems that selecting two or more
> > things can mean different things:
> >
> > 1) it can mean you are doing two things.
> > 2) it can mean you aren't doing anything and you want to do
> something.
> >
> > But, you can't say you are currently outsourcing, but want to do it
> in
> > house
> >
> > I wonder if this is better as two questions?
> >
> > Or, move the Would like to add to our digital preservation program
> > first, and add in a new column for "Would like to do in-house"?
> >
> > On Q16: should we make it "and/or", rather than just or?
> >
> > The survey looks great!
> >
> > ~ Amy
> >
> > -----Original Message-----
> > From: The NDSA Standards working group list [mailto:NDSA-
> > [log in to unmask]] On Behalf Of Goethals,
> > Andrea
> > Sent: Monday, May 21, 2012 2:13 PM
> > To: [log in to unmask]
> > Subject: [NDSA-STANDARDS] please review revised version of the
> > staffing survey
> >
> > Hello,
> >
> > I made some revisions to the dp staffing survey based on:
> > - feedback from our last WG call (defines FTE, gives example of
> > other department, ability to indicate that an area isn't considered
> > part of the dp function)
> > - feedback from Mary Vardigan via email (primarily about the
> > privacy text, but also some new options for the type of
> > organization, and the wording of the question about the amount of
> > files being preserved)
> > - best practice videos/manuals on building surveys (by Qualtrics and
> > SurveyMonkey) Based on these I simplified how the number ranges were
> > presented for the question on amount of content, simplified some
> > question text, and added some more N/A and Other options.
> > - capability of Qualtrics - using a feature called "carry forward
> > choices", they're now asked which areas are considered part of the
> > dp function at their org, and in a follow up question they are asked
> > the status of any they had selected in the previous question.
> >
> > You can try out a test version of the survey with this link: Try
> > the test version of the
> > survey.<https://harvard.qualtrics.com/SE/?SID=SV_7PdihNMVB9FnT6I>
> > Currently 2 of the questions are required (name of organization,
> > type of organization) so you won't be able to advance to page 2 if
> > you leave these blank.
> >
> > Any and all feedback is welcome.
> >
> > For the "real" survey, Jimi and I were thinking that a month should
> be
> > sufficient time to keep the survey open once we're ready to publish
> it.
> > That should give us enough time to look at the number of responses
> > coming in and send out reminders if needed. Any objections to the
> > month length?
> >
> > Thanks,
> > Andrea
> >
> > Andrea Goethals
> > Digital Preservation and Repository Services Manager Harvard Library
> > [log in to unmask]
> > (617) 495-3724
> >
> >
> > ############################
> >
> > To unsubscribe from the NDSA-STANDARDS list:
> > write to: mailto:NDSA-STANDARDS-SIGNOFF-
> > [log in to unmask]
> > or click the following link:
> > http://list.digitalpreservation.gov/SCRIPTS/WA-
> DIGITAL.EXE?SUBED1=NDSA
> > -
> > STANDARDS&A=1
> >
> > ############################
> >
> > To unsubscribe from the NDSA-STANDARDS list:
> > write to: mailto:NDSA-STANDARDS-SIGNOFF-
> > [log in to unmask]
> > or click the following link:
> > http://list.digitalpreservation.gov/SCRIPTS/WA-
> DIGITAL.EXE?SUBED1=NDSA
> > -
> > STANDARDS&A=1
>
> ############################
>
> To unsubscribe from the NDSA-STANDARDS list:
> write to: mailto:NDSA-STANDARDS-SIGNOFF-
> [log in to unmask]
> or click the following link:
> http://list.digitalpreservation.gov/SCRIPTS/WA-DIGITAL.EXE?SUBED1=NDSA
> -
> STANDARDS&A=1
>
> ############################
>
> To unsubscribe from the NDSA-STANDARDS list:
> write to: mailto:NDSA-STANDARDS-SIGNOFF-
> [log in to unmask]
> or click the following link:
> http://list.digitalpreservation.gov/SCRIPTS/WA-DIGITAL.EXE?SUBED1=NDSA
> -
> STANDARDS&A=1
############################
To unsubscribe from the NDSA-STANDARDS list:
write to:
mailto:[log in to unmask]
or click the following link:
http://list.digitalpreservation.gov/SCRIPTS/WA-DIGITAL.EXE?SUBED1=NDSA-S
TANDARDS&A=1
############################
To unsubscribe from the NDSA-STANDARDS list:
write to:
mailto:[log in to unmask]
or click the following link:
http://list.digitalpreservation.gov/SCRIPTS/WA-DIGITAL.EXE?SUBED1=NDSA-S
TANDARDS&A=1
############################
To unsubscribe from the NDSA-STANDARDS list:
write to: mailto:[log in to unmask]
or click the following link:
http://list.digitalpreservation.gov/SCRIPTS/WA-DIGITAL.EXE?SUBED1=NDSA-STANDARDS&A=1
