Code-sharing policy: update

More and more articles coming out in the JF now including replication code in their supporting information. The current issue (August 2018) includes three articles with code:

https://onlinelibrary.wiley.com/doi/abs/10.1111/jofi.12698

https://onlinelibrary.wiley.com/doi/abs/10.1111/jofi.12688

https://onlinelibrary.wiley.com/doi/abs/10.1111/jofi.12697

Two of these three teams of authors submitted the code voluntarily even though the paper was not formally subject to the code sharing policy (the authors had made the initial submission of their paper before we adopted the code sharing policy in September 2016).

Many thanks to these authors for going the extra mile to enhance reproducability of research in our field!

“Reject-and-resubmits” — why we discontinued them at the JF

Editorial decisions traditionally come in three forms: reject, revise and resubmit, and accept. Over the years, an additional category appears to have become more common: “reject and resubmit.” In a typical reject-and-resubmit, the editor tells the authors that even though the current submission is formally rejected, the authors have the option of submitting an “essentially new” paper that may include parts of the rejected submission. Formally, the resubmission is treated as a “new submission.” What often happens in practice is that the authors submit a heavily revised version of the earlier submission along with replies to referees and the editor, just as they would when resubmitting a paper that received a revise and resubmit.

In 2016, we discontinued the use of “reject-and-resubmit” at the Journal of Finance. After giving the issue some thought, we came to the conclusion that “rejects-and-resubmits” are detrimental to a transparent and predictable editorial process:

1. Since “reject-and-resubmit” does not exist as a formal decision category (formally, it’s a reject after all), it can be unclear to authors, especially inexperienced members of our profession, how to proceed: What sort of paper are they expected to resubmit? Should they prepare a response to referees?

2. “Reject-and-resubmits” generate unnecessary uncertainty about the process ahead: Will the resubmission go to the same referees or even the same editor? What happens when the editorial team changes?

3. Looking at a journal’s reported statistics, how many of the papers that a journal reports as rejected are actually “reject and resubmits”? How many of the papers that are reported as accepted after two revisions are actually ones that had received a “reject and resubmit” in “round zero” and are therefore, effectively, papers that required three rounds of revisions? What percentage of “new” first-round submissions are actually double-counted resubmissions of “reject and resubmits”? Since standard editorial software does not keep track of the (formally non-existing) “reject-and-resubmit” decisions, it can be difficult even for editors to figure out precise answers to these questions.

4. Having the option of issuing a “reject-and-resubmit” can tempt an editor faced with a tough case to “kick the can down the road,” delaying a resolution.

To be clear, there is an important role for speculative revise and resubmits for papers that have a promising core but substantial challenges to overcome in order to clear the bar for publication. We regularly issue such speculative revise and resubmit decisions. Since these are treated formally as revise-and-resubmits, the process is clear: Typically, the resubmission will go back to the same referees and the same editor will handle the resubmission.

Code-sharing policy: update

In September 2016, the JF adopted a code-sharing policy. The first accepted papers with replication code are now online (scroll down and click on supporting information):

http://onlinelibrary.wiley.com/doi/10.1111/jofi.12615/full

http://onlinelibrary.wiley.com/doi/10.1111/jofi.12620/full

Reproducibility of research findings is a basic principle of science and a pre-requisite for replicability. Many researchers have undoubtedly had the experience of trying, unsuccessfully or with great difficulty, to reproduce the results of a published paper. In most cases, the authors of the published paper cannot be blamed for these difficulties. Many empirical and computational research projects these days are complex. Files are merged from many data sets, data manipulation is extensive, and empirical techniques have become more and more sophisticated. It has become virtually impossible to catalog all the precise data definitions and specifications in the paper or an online appendix. This is where code sharing can help.

We strive for the JF to be a journal that meets the highest standards of quality, reproducibility and replicability. By agreeing to share the code, authors submitting to the JF signal that they, too, are committed to these high standards. With code transparency, research findings will be more credible and they will, ultimately, have greater impact.