Output – RIOJA – Survey: publish where?

Title: RIOJA (Repository Interface to Journal Archives): results from an online questionnaire survey

Page: 33

Summary of contents:

The 683 astrophysics researchers responding to the survey were asked about factors that would encourage them to publish in an overlay journal. “In line with responses discussed in previous sections (e.g.,Table 2), the scientists emphasised once again that the most important factor that would encourage them to publish in any journal is the quality of the other submitted papers (526 people, 77% of base=683). This result could be interpreted as directly associated with the following two factors: the transparency of the peer review process (410 people, 60% of base=683) and next the reputation of the editor/editorial board (386 people, 57%, Figure 19). Response by subject area and role of the respondents, though, clearly puts the emphasis on the role, integrity and experience of the editorial board (Table 11, Table 12). In comments, the respondents noted several other factors that would encourage them to publish in an overlay journal. Those listed included references to the impact factor of the journal, the acceptance of the journal by the community, and acquiring a reputation as a quality, scientific publication with increased readership and breadth of coverage. The quality of the refereeing and some guarantee of open access and low charges were also mentioned (comments in full are listed in Appendix B, on page 81).”


This survey provides an indication of what factors would encourage the surveyed of researchers to publish in overlay journals. The finding provides evidence that any new form of scholarly communation still has to fulfill the functions that existing channels provide.

Date Released: February 2008

URI for Output: http://eprints.ucl.ac.uk/5102/1/RIOJA_questionnaire_survey_report_final.pdf

Output – RIOJA – Survey: who pays?

Title: RIOJA (Repository Interface to Journal Archives): results from an online questionnaire survey

Page: 33
Summary of contents:
The 683 astrophysics researchers responding to the survey were asked about paying for the costs associated with running an overlay journal.

“When asked where the funding to meet those costs should come from, the respondents preferred to select research funders (485 people, 71% of base=683), library subscriptions (432 people, 63%) and sponsorship, for example by a Learned Society (350 people, 51%). A model requiring an author to pay from research funds either on acceptance (218 people) or on submission (47 people) of a paper was endorsed. Other sources mentioned in comments included: personal donations, professional association contributions, commercial and/or not-for-profit organisations, advertisements, subscriptions and even models of having authors pay partially on submission and partially acceptance. Although the general trend picked out research funders, responses from different subject groups tended to pick out library subscriptions (Table 9).”

This survey provides an indication of how a number of researchers feel journals/ innovations in scholarly communication should be funded (corresponding infrastructure costs are presumably part of this general cost).

Date Released: February 2008

URI for Output: http://eprints.ucl.ac.uk/5102/1/RIOJA_questionnaire_survey_report_final.pdf

Output -RIOJA -Costs and sustainability: overlay journal costs

Title: Repository Interface for Overlaid Journal Archives:costs estimates and sustainability issues

Pages: 12-16
Summary of contents:

The report sets out the costs for each of the four identified core functions of a journal:


first copy costs – editorial board, assigning reviewers etc., support and admin –
“Consultants in SQW Limited (2004) reported that first copy costs for a good to high quality journal are
estimated around – average price – $1500 ($1650 including first copy and fixed costs).”
however, “Harnad (2000) …indicates that conducting the peer review electronically and for papers residing in an open access archive could cost about 1/3 less of the actual page cost.”
ArXiv uses a low cost system of endorsement ($1-5 per item) in which previous submitters vouch for the relevancy of new work.

p13-p14 Note survey finding that there is no consensus on the issue of open or closed peer review.
p14 “King & Tenopir (2000) list the following activities in article processing: manuscript receipt processing, initial disposition decision making, identifying reviewers or referees, review processing, subject editing, special graphic and other preparation, formatting, copy editing, processing author approval, indexing, coding. There are also significant indirect costs – costs not directly associated with a particular process, such as administrative and managerial costs. Rowland reports costs for peer review in the range of $200-$400 per paper, including administrative support, for a journal with rejection rate of 50%.”

p14-15 This covers current awareness and related dissemination tools and activities. The survey has noted the importance of such functions to the community using arXiv. RIOJA comments p15 “The awareness functions provided by arXiv and other repositories could clearly reduce central overheads for a repository-overlaid journal.” (see comment)

This section briefly discusses how an overlay journal would need to ensure the preservation of accepted content. The section has little specific discussion of the practicalities and problems of preservation outside of the context of arXiv but does make a very useful suggestion in that on demand printing services are available ‘for printing paper versions of the journal’s issues at a cost of less than $250, including shipping and handling.’ (p16)

Awareness – while it’s true that a repositories alerting services could reduce the need for an overlay journal to have such services, there is a tnesion here that the project doesn’t note (afaik) – not offering these services would significantly reduce the overlay journal’s visibility/ identity. not offering these services could have a direct impact on the visibiilty and ‘impact’ of the the journal. However, in the context of a journal based on items from a single repository (such as arxiv) the point is well made that this service is carried out anyway. [not clear if project thinks this though]

Archiving – though this costing largely sidesteps the issue of digital preservation the suggestion that (at least in the short term) copies of record could be printed on demand appears a significantly less expensive option than the current printing process.

Throughout the costing there is a heavy reliance on the ‘unique’ / ‘mature’ context of arXiv. It is not yet clear which of these characteristics of arXiv has had the stronger effect.

This examination of costing contributes to identifying relevant shared infrastructure services and assessing their feasibility.

Date Released: July 2008

URI for Output: http://eprints.ucl.ac.uk/12562/1/12562.pdf

Output -RIOJA -Costs and sustainability: overlay journal model

Title: Repository Interface for Overlaid Journal Archives:costs estimates and sustainability issues

Pages: 4-6
Summary of contents:

dftn: “Overlay journal – For the purposes of this report an overlay journal is defined as a quality-assured journal whose content is deposited to and resides in one or more open access repositories.”

p5-6 review the literature on the idea of an open access overlay or deconstructed journal.

Functions of journal publishing are delineated – any emerging model needs to address these:
“Journals are traditionally held to perform four “first order” functions (Meadows, 1974;
Rowland [2002], Roosendaal and Geurts (1997) as cited by Prosser (2005)):

  • Registration: an author wishes to be acknowledged as the person who carried out a specific piece of research and made a specific discovery
  • Certification: the author’s claims are tested through independent peer review, and it is determined that they are reasonable
  • Awareness: the research is communicated to the author’s peer group
  • Archiving: the research is retained for posterity

To those mentioned above Prosser adds the function of ‘Reward’ to the author.”
“Prosser, David C. (2005) Fulfilling the promise of scholarly communication – a
comparison between old and new access models, in Nielsen, Erland Kolding and Saur,
Klaus G. and Ceynowa, Klaus, Eds. Die innovative Bibliothek : Elmar Mittler zum
65.Geburtstag, pp. 95-106. K G Saur. (Also available at
http://eprints.rclis.org/archive/00003918) (Last accessed 31/07/2008)”

note: The type of overlay journal proposed is one involving original submission to the overlay journal (simultaneous to or concurrent with submission to repository). The issue of overlay journals drawing on work submitted to other journals is not addressed/ in scope.

relevancy to categories
Any component of an emergent infrastructure wanting to be journal-like needs to consider these issues – as such they provide a baseline that RIOJA (and others) are going to address. As the survey results (noted elsewhere) indicate motivating academic researchers to use other forms of publication needs to interact with these issues.

Date Released: July 2008

URI for Output: http://eprints.ucl.ac.uk/12562/1/12562.pdf

Output – RIOJA – Final Report: researchers’ views

Title: RIOJA (Repository Interface to Journal Archives) Final Report

Page: 5 (summary), p11 (outputs and results1: overview)
Summary of contents:

p5 “The survey confirmed the everyday importance of the arXiv repository in the working lives of astrophysics researchers. Moreover, the project found that researchers are, in general (and with very little variation between those with different first languages, career lengths and other demographics), sympathetic to the overlay model. Their main concerns about the model were that the long-term accessibility of the research material should be guaranteed – surprising, perhaps, in such a fast-moving, repository-dependent discipline – and that the process of quality certification should be robust. Researchers’ career concerns also
informed their reaction to the overlay model, and it was clear that to attract submissions, an arXiv-overlay journal would need to be able to demonstrate academic acceptability and a substantial readership. All of these concerns are generic issues, which would be faced by any new journal whether or not overlaid on repository-housed content.”

p11 “The supporting survey gave a snapshot of the working practices and attitudes of one, very repository-orientated, research community, based on 683 responses from Astrophysicists. The results confirmed the importance of arXiv to Astrophysics researchers. 93% deposit papers into arXiv; 53% access arXiv daily, and another 24% do so weekly; and after arXiv discovery, only 7% always prefer to seek the final published version of a paper. arXiv use is not to the exclusion of other resources: 65% may use journal Web sites to follow up interesting titles/abstracts, alongside arXiv which is used by 610 (89%) for this purpose. 97% of the respondents publish in refereed journals, at an average of 6.5 papers per researcher per year, in titles whose high impact factor, perceived quality, and updates throughout the refereeing process they consider to be important. They were comfortable with the overlay model: 53% were very supportive, and 35% interested; 80% would referee for an overlay journal; 26% were willing to serve in Editorial capacity; 33% would submit papers without hesitation. Their concerns about a hypothetical arXiv-overlay journal were the quality of the accepted papers, the community standing for the title, the robustness of long-term archiving arrangements, and the quality and speed of of the peer review process: these are concerns which one might imagine could easily apply to any academic journal, regardless of publishing model.”


reflection: it is unclear how many papers go into arxiv without an intended journal? it seems that part of the premise of the way this is pitched is that submission to arXiv in itself makes something worth reading… but the feedback strongly suggests that peer review/ journal quality is key. This may indicate there’s a very strong self – moderation going on – researchers can rely on arXiv copies because ‘no-one’ puts anything in that can’t make the grade… comments elsewhere indicate that papers being put into arXiv are ‘endorsed’ by exisiting ‘senior’ contributors, but judging by these results this ‘endorsement’ is only serving as a stop gap prior to formal peer review.

This survey data directly informs our understanding of one community of researchers around one repository. The repository, arXiv, and community, astrophysics, have been at the forefront of the use of repositories to support scholarly communications. The project’s findings indicate that any new form (or brand) of publication still needs to demonstrate some form of impact factor and stringent quality control process if it is to sucessfully engage researchers.

Date Released: September 2008

URI for Output: http://eprints.ucl.ac.uk/12562/1/12562.pdf

Output – RIOJA – Journal Repository APIs

Title: RIOJA Journal-Repository APIs

Page: web page with related files

Summary of contents:
The page contains an introduction to and details about the xml APIs developed by RIOJA.
The APIs cover:
” Repository APIs

  • getRepositoryInfo: Get information about the repository
  • validateAuthor: Author validation
  • getMetadata: Metadata exchange
  • setStatus: Journal status change notification
  • getStatus: Get journal status from repository
  • getTrackbacks: Get repository trackbacks
  • getStatistics: Get repository statistics
  • getCurrentPaperInfo: Get current state of paper

Journal APIs

  • getJournalInfo: Get information about the journal
  • getStatus: Get submission status within the journal
  • submit: Submit to journal from repository “

The final section is of note:

  • Author managed papers are assumed by these APIs.
  • ‘Why not just use OAI?
  1. OAI is designed for metadata harvesting not as an immediate-reponse API to be used interactively. For RIOJA journals need to be able to get metadata dynamically during the submission process; OAI allows “wait” responses which require re-querying at a later time.
  2. RIOJA allows for extraction of information before a paper is public on a repository (e.g. for integrated submission) before it would be available by OAI
  3. RIOJA needs to track paper versions carefully. OAI does not include detailed paper version information.
    There should be a well-defined format for titles and abstracts with text formatting and equations so that they can display the same way on the journal and repository without further editing; OAI is very flexible but very vague.
  4. RIOJA requires several other functions apart from extracting Metadata ‘

This rationale is also laid out in the final report page 9.http://eprints.ucl.ac.uk/12562/1/12562.pdf
This section also details similarities between some of RIOJA’s desired functionality and that subsequently developed in SWORD.


As commented on the software – the API’s currently require custom versions – if this functionality doesn’t make it into a core release using/ supporting these API’s is unlikely to be sustainable.

The case to replace an accepted api has to be very strong.

Looking at these reasons OAI-PMH is deemed unsuitable:

  1. no1 – In can see the case for this but given the model of overlay journal that the project has chosen (simlutaneous deposit to repository and for overlay publication). Although I’m not convinced this is the only possible workflow – this reason in itself is probably sufficient to examine alternative apis.
  2. no.2  – this seems to be more to do with repository configuration than OAI-PMH as such. There is no reason that a repository can’t be configured for secure OAI-PMH metadata harvesting. Again a lot is predicated on a particular workflow.
  3. no.3 – this is just plain wrong – versioning information is a question that relates to bibliographic metadata, OAI-PMH is a harvesting protocol. You can put whatever information you want into OAI-PMH. The observation is true for the simple dublin core metadata (oai_dc) that you must include with  a oai-pmh compliant repository – but this does not prevent you adding any other sets of information. It’s like saying you need an alternative to roads because your car needs lpg not petrol. It is certainly true that the information you get from most existing repositories (via OAI-PMH) does not support the needs of the an overlay journal but this, in itself, is not a reason to replace OAI-PMH.
  4. no4 – impossible to comment.

I’m not suggesting that there isn’t a need to use something other than OAI-PMH for an overlay journal, but, aside from the first point (which could be significant enough in itself, but is predicated on a particular workflow) the summary arguments for implementing an entirely new api interface are are weak. In the Final Report possible overlap with SWORD (which developed independently while the project was running) is noted. It doesn’t examine SRW/SRU interfaces.

I’d suggest this is a good example of an infrastructure encountering the limits of accepted protocols but also of a development setting itself up for long term problems by effectively requiring 3rd party support for custom api’s [though, please note: this is not a comment on the effectiveness of these APIs for their purpose as a pilot/ proof of concept service]

Date Released: 2008

URI for Output: http://cosmologist.info/xml/APIs.html

Output – RIOJA – Overlay Journal software products

Title: RIOJA Project Software Products

Page: web page with related files

Summary of contents: the page points to downloads for ‘modified software packages to start journals based on existing repositories, and to start new API-enabled ePrints-based repositories.’

The software is a modified version of OJS journal system (2.1.1). Modifications include:
‘ * Quick validated submission of papers using existing repository IDs
* Keyword-based matching system to speed assignment of referees and editors
* Continuous publication, using links to accepted versions of the paper hosted on the repository.
* Removal of parts not needed in an overlay journal (print publication, etc) ‘

The software retains OJS’ OAI functionality. As it stands the software can be used with the Arxiv repository or any modified ePrints installation (a modified version of ePrints 3 is also available for download).

Sample journals based on the software is linked: http://arxivjournal.org/ . Of the journals on this page the first two are linked explicitly to the project – cosmology continous and cosmology issued. Cosmology continous has a current issue (from July 2008) none of the other pages associated with these journals have any content. The other four journals seem to be experimental only either having no content or being labelled for developers and requiring a login.

There are a number of outstanding bugs noted.

A model of software development that relies upon releasing modified versions of software both for data provider and overlay journal is probably going to struggle to be sustained and adopted.
From what I can tell RIOJA has demonstrated the concept of an overlay journal in this field well but unless their work makes it into core releases of the software they’re adopting (an issue not discussed here) their work will remain as proof of concept.

The issues encountered in this software development are relevant to any infrastructure project developing or heavily moddifying software.

Date Released: 2008

URI for Output: http://arxivjournal.org/rioja/

Project – Rioja

Repository Interface for Overlaid Journal Archives

Short Project Name: RIOJA

Programme Name: Repositories and Preservation Programme

Strand:Tools and Innovation
JISC Project URI:
Project URI: http://www.ucl.ac.uk/ls/rioja/

Brief project description:
” The RIOJA project will investigate the overlay of quality assurance onto papers deposited to and stored in eprints repositories… The partner institutions are:
* UCL [lead]
* University of Cambridge
* Cornell University
* University of Glasgow
* Imperial College London

RIOJA will create a tool to support automated interactions between journal software and public repositories. The project will also build a pilot “overlay journal”, which will demonstrate interactions between the arXiv subject repository and OJS journal software, facilitated by the RIOJA tool. Additionally, RIOJA will explore some of the social and economic aspects of building certification onto repositories. A large-scale survey of researchers from the field of Astrophysics and Cosmology will be carried out, and a feasibility study on the costs and sustainability of an overlay journal in this field will be published. ”


  1. RIOJA software outputs http://arxivjournal.org/rioja/
  2. RIOJA Project final report http://eprints.ucl.ac.uk/12562/
  3. Repository Interface for Overlaid Journal Archives: costs estimates and sustainability issues http://eprints.ucl.ac.uk/11927/
  4. Repository Interface for Overlaid Journal Archives: results from an online questionnaire survey http://eprints.ucl.ac.uk/5102/
  5. Rioja APIs http://cosmologist.info/xml/APIs.html
  6. There is also an extensive list of publications http://www.ucl.ac.uk/ls/rioja/dissem/

Name of Trawler: John
Start Date:2007-03-01

End Date:2008-06-30

Governance:Integrated Information Environment Committee (JIIE)

Contact Name and Role: Martin Moyle Project Manager