Hi all,

I've done a couple projects mining the data from the code4lib listserv
(e.g. https://ejournals.bc.edu/index.php/ital/article/view/5893 ).  Both
times the fastest route was finding helpful folks involved in it to provide
me with a data dump vs. spending time on a scraper.

The most recent work I did was in 2018 - I have a tarball of all the
message log files for the listserv (some will be job posts and others not)
which is 2003 through 2018.  I believe I asked about this on the c4l Slack
at the time and Wayne Graham from CLIR kindly helped me out with the data!
This data is not anonymized (as it was/is publically available with names
and emails associated) but I did anonymize the findings for reporting.

Ellen - I'd be happy to chat sometime about how I mined the data for job
titles and related skills/technologies, feel free to reach out to me
directly!

Best,

Monica Maceli, Ph.D.
Associate Professor
Pratt Institute | School of Information
144 W 14th St, 6th Floor, New York, NY, 10011-7301
www.monicamaceli.com | [log in to unmask]


On Fri, Jan 22, 2021 at 1:18 PM Andromeda Yelton <[log in to unmask]>
wrote:

> The initial commit in https://github.com/code4lib/shortimer/ was November
> 2011, which is ten years for some values of ten. Taking a quick and
> noncomprehensive glance around, I see postings as old as 2005. I don't see
> an obvious API, but maybe a maintainer could weigh in about data dump
> possibilities?
>
> On Fri, Jan 22, 2021 at 11:28 AM Eric Lease Morgan <[log in to unmask]> wrote:
>
> > On Jan 22, 2021, at 11:11 AM, Jill Ellern <[log in to unmask]> wrote:
> >
> > > I'm doing some research into systems librarian duties and wondering if
> > there is an easy way to get a dump of the code4lib jobs from the last 10
> > years?  In excel format?
> >
> >
> > Easy? I'd be surprised.
> >
> > There are two or three sources of the Code4Lib jobs data:
> >
> >   1. the underlying data from the jobs.code4lib.org site
> >
> >   2. any one of a number of different Code4Lib mailing list Web archives
> >
> >   3. the archived mailbox (mbox) files from the mailing list
> >
> > I don't think the jobs site has been around for ten years. Has it? Nor do
> > I know whether or not the data is archived. If it is, then I'd bet you
> will
> > be able get it in some sort of structured format like JSON or delimited
> > delimited format like Excel.
> >
> > Scraping different Web archives would require... scraping which,
> > personally, I run away from.
> >
> > Finally, the archived mbox files would be the most comprehensive, but a
> > programmer would have to parse the mbox (email) files, which is a
> > specialized task in and of itself. If you want to know where the mbox
> files
> > are located, then drop me a line and I'll let you know. Easy.
> >
> > Finally, what's the questions you would like to answer? How many system
> > librarian jobs have been posted? Where were the jobs? What are the
> > characteristics of systems librarianship and how have they changed over
> > time? How much they pay? Extracting some of this information from the
> > postings may be difficult, if not heroic in nature.
> >
> > --
> > Eric Morgan
> > University of Notre Dame
>
>
>
> --
> Andromeda Yelton
> Humanistic Machine Learning for Library Data
> Lecturer, San José State University iSchool
> https://andromedayelton.com
> @ThatAndromeda
> <http://twitter.com/ThatAndromeda>
>