Jack Gallant

@DrAnneCarpenter @benbfly @aexbrown @manlius84 @nature I'm not deflecting I have been consistent in all these threads. IMHO the decision should be based the complexity of the data and the methods that need to be developed to analyze it. One size doesn't fit all. Treating everything the same will have unintended negative consequences.

@benbfly @gallantlab @aexbrown @manlius84 @nature His deflection about trainees and disincentivizing is so strange. If these were important issues, then follow the policy and state this when submitting the work to Nature. Don't claim you will release data promptly then refuse to. We seem to be unable to find common ground.

If you are still writing your code in Matlab, please try to transition to a real open language like Python. Matlab is NOT open code, it is a walled garden. If you believe in open science then you shouldn't be using it. (As it happens Matlab is also poorly suited to big projects.)

@DrAnneCarpenter @benbfly @aexbrown @manlius84 @nature Sorry Anne I wonder (1) why you don't seem to be concerned about protecting young students who collected the data and are still working on primary papers and (2) why you think immediate data dumps won't disincentivize big projects. Every decision/rule involves tradeoffs.

@gallantlab @benbfly @aexbrown @manlius84 @nature I'm not angry and my name is not Ann. You should have included the statement "We will not be releasing data promptly. We will release the data publicly after publishing N additional studies." It's astonishing to me that you seem to genuinely think this suits Nature policy.

@benbfly @DrAnneCarpenter @aexbrown @manlius84 @nature All of our data are released on crcns.org. These data will be released on crcns.org. And the code for our whole pipeline is being released piecemeal as we make it usable for other humans, that is on our github page. We're not "hoarding" anything.

@benbfly @DrAnneCarpenter @aexbrown @manlius84 @nature I understand but I don't think that you're fully considering the differences in data taken in different fields/labs, or the impacts on students who take a big risk in doing something really ambitious and novel that can support multiple papers but only after many years of work.

@lmeyerov @manlius84 @aexbrown @nature @imerov I am one of those open source folks! We release lots of code on github, including the best brain viewer out there right now. And our data are released on crcns.org. However, the timelines in CS and neuroscience are inevitably different.

@manlius84 @gallantlab @aexbrown @nature Imagine if all us open source folks you're building on also said, "We release all our code, after our years of primary... " 🙄 On CS side @ berkeley, we put out data+code as part of publishing the reproducible artifact and contribution to tax payers and fellow explorers.

@CaAl @maltoesermalte @nicebread303 @lakens @sharoz But again, all the data are being used in these papers. My lab has a longstanding policy of releasing all data once the primary papers are out, and we have a clear track record of doing this. You can check crcns.org for our data or github for code.

Interesting discussion on whether labs should share costly to collect datasets after publication, or whether they can keep the data for themselves for several years to publish more papers from it. I think @gallantlab should share this data. twitter.com/manlius84/stat…

@DrAnneCarpenter @aexbrown @manlius84 @nature You seem pretty angry Ann but FYI 1) our simple data policy is that we release ALL data after the primary papers are out, 2) this is to protect the students who collected the data and developed the methods, 3) doing otherwise would disincentivize highly novel/complicated studies.

@gallantlab @aexbrown @manlius84 @nature And you're deluding yourself if you think the word "prompt" is open to interpretation. @nature should clarify the term to avoid people using this excuse.

@rubenarslan @lakens @sharoz I think the field is moving toward that as an alternative venue for credit assignment and I think that is a good idea in general. Then people could cite the primary research papers or the data as appropriate.

@gallantlab @lakens @sharoz The UK biobank seems to work well.. but that this is not your situation. Would you release more quickly if data were more like papers, citations to which went into standard author metrics, maybe including even a pagerank factor (ie. your data led to highly cited pub in glam mag)?

@gallantlab @sharoz I understand, and you are right that if we give single labs more money than other labs to collect costly data (instead of organizing data collection collaboratively) we need to think how much people should get rewarded (not you already were rewarded with money to collect data).

@DrAnneCarpenter @aexbrown @manlius84 @nature I am sorry that you feel that way Ann. We have always been up front about our data sharing policy. We share all data on crcns.org after the primary papers have been published. This is to protect the students who collected the data until they can finish their work.

@gallantlab @aexbrown @manlius84 @nature I've no skin in this game but you are blatantly not abiding by Nature's policy. Your motivations for breaking their rules are irrelevant. If you intended to keep the data private for one year this should have been stated to reviewers and in the article.

@nicebread303 @lakens @sharoz That seems like a really weird solution to the problem. If the data are completely novel and require new methods of analysis then the timeline is unknown. A better solution would be to base future funding on past evidence that data collected were eventually shared.

@nicebread303 @lakens @sharoz I argued that it should depend on the complexity (information content) of the data and the extent to which new analyses will be required to publish subsequent papers. I don't think that there is a one-size-fits-all rule. But 5 years certainly seems generous.

@jjolij @jjfahrenfort @lakens This sounds like a good policy, especially if there are boilerplate license forms available for various scenarios. It would protect the students while getting the data out quickly. However, AFAIK we have nothing like that over in the US.

@jjfahrenfort @lakens @gallantlab Under EU law, IP of datasets remain with the processor (ie institute where data was collected). Data licensing can be done and is not uncommon for PhD-thesis data to make sure a PhD student has 'first right' to publish on findings. It's part of RUG PhD protocols, for example.

@aexbrown @NeuroPolarbear @BorisBarbour @manlius84 Again we all agree that the data should be released. The question is when. Immediate release has negative repercussions for the students who collected it and disincentivizes large studies. There is no perfect solution.

@NeuroPolarbear @BorisBarbour @gallantlab @manlius84 Because public data has a third important effect which is that #researchparasites integrate it with other data and publish studies the original lab would never have done or doesn’t have the skills to do.

@lakens @sharoz Big data sets collected by consortia won't solve the problem. They are often "designed by committee" which means they are mediocre for everything and good for nothing. That said, I would LOVE it if someone would collect a large data set for me, as long as it met my standards!

@lakens @sharoz People doing quick cutesy studies that are significant but which have small effects is precisely what got us into this replication problem to begin with! We should be incentivizing high-quality science, and you can't do that quickly.

@lakens @sharoz Look Daniel the bottom line is that if everyone has to release all data on first publication then people will be incentivized to collect small, low-information-content data sets. You know that and I know that. And that will not be good for science.

@RomainBrette @manlius84 @aexbrown @nature We are obeying the rules. We are getting this done as fast as possible. We have a long track record of releasing data and we will release these data too just like we always do. I feel like you are just trolling now...