Wikiversity talk:IRC meeting about research

So wait, what time zone am I? It's currently 12 am here--Rayc 06:09, 22 February 2007 (UTC)

My computer has a time control panel that can help figure this out. Also useful: World clock online. --JWSchmidt 15:39, 22 February 2007 (UTC)

Or if you prefer a graphical display, try this page. Oh, and to answer Rayc's question - your timestamp came up as 06:09 UTC, so you're 6 hours behind UTC - which you could confirm at wikipedia:Time in the United States. Cormaggio 19:12, 23 February 2007 (UTC)

• Cormaggio: wikipedia:en:Time in the United States--Hillgentleman|書

Synopsis

Anyone want to do an abridged version of this? I'll give it a try:

• Intros
• What is the difference between Secondary and primary (New ideas from old data vs. New data)?
• Uniform policy, good or bad thing?
• Who's law applies to us?
• What research should we reject?
• Splits:
  • Who is "we"? (Review board... who is the review board?)
  • Review board: Experts in fields, or experts on policy?
  • Expert verification
• &
  • Publishing copyright concerns (Can dual license stuff be published?)
  • Should we be the publisher or someone else?
• Recombined into:
• Should "we" (review board) say what research is worthwhile, or should the publisher?
• Vandalism in research
• How do we get researchers to put research on WV? (Trolls->Researchers?)
• Expert !=PhD
• Resaerch: Inclusive, with caveats, or Only educational related?
• Splits:
  • What is this meeting purpose?
  • What are the steps for setting up research, and where does the review board come in?
  • Reviewboard approves research first, or deals with it like AfD?
• &
  • What to do with crackpots?
  • Do we want a flood of researchers, and how will small wikis handle it?
• Recombines into:
• Are we looking to far ahead?
• No research until a wiki gets a Referees?
• Referees=Experts!=PhD again
• How do we get academics? (again)
• All research educational on how to do research, thus all research education research? (again)
• Bias in research (Is all researched biased?)
• Referee = Expert, take 3
• Request for deletion like system, take 2
• Ask the quiet people their opinion, say night to others
• Purpose of the meeting, part 2
• Close meeting. --Rayc 23:32, 11 March 2007 (UTC)

Notes

I'm not sure if it is of much use to others, but my review notes on the discussion are at User:JWSchmidt/Blog/11 March 2007

People who were there

Of the 30 or so people there, 17 introduced themselves (if anyone with unicode logs could fix up #14, I'd be happy)

1. WiseWoman, professor for Media and Computing in Berlin, jack-of-all-trades, American by birth, more or less disillusioned by Wikiversity at the moment, still playing with diverse Wiki-ideas.
2. Michael Billington, Australian Wikiversity custodian, general wiki-enthusiast, and programmer.
3. mikeu, astronomer at Ladd Observatory, Providence RI. inerested in informal education.
4. Ward Cunningham, author of first wiki, now building model of wiki community behavior, and therefore paying attention to other research in the area.
5. GBoyers (Xenon), computing student in the UK, research enthusiast and policy-junkie.
6. Rayc, American Wikiversity custodian, master's student, with wiki-Test and Quiz interests.
7. Kat (mindspillage), Wikimedia board member, JD student, would like to never leave university. :-)
8. Javier Carro, spaniard living in Zurich, I just begin my doctor on applied linguistics. I'd like to research on wiki-based language learning. But I still need to clarify concepts, steps to give forward...
9. Cormac Lawler ("cormaggio"), researcher at University of Manchester (school of education) - researching the progress of Wikiversity as an educational community/resource/...
10. John Schmidt, biologists, exploring the power of wiki
11. Haikon, living in US, no higher education, interests in facilitation those with higher education
12. Greg Maxwell, Chief Research Coordinator for the Wikimedia Foundation. (research is primarily referring to research about Wikimedia projects, not projects performing research). I'm here just to keep updated on Wikiversity and to keep an eye out for people I can help.
13. yeoman (CQ), electrician/musician novice programmer from Kentucky USA (now in Illinois).. interests: community-building, sustainabilty, ecovillages, web radio... Enthused with Wikiversity :)
14. jtico, es-wiki editor and Professor at the Universidad Simón Bolívar http://cronos.ma.usb.ve
15. Strangerer, Undergraduate, a senior in mathematics at the University of Illinois at Urbana-Champaign with a concentration in Aerospace Engineering.
16. AngryParsley I'm a software engineer with interests in pretty much anything computer-related
17. HistoryBuff, Like history, but computer stuff is my profession.

Some things to note:

• We had the inventor of wiki there.
• As well as a steward (and a second one was there also, though she didn't speak)
• A member of the Wikimedia board
• Wikimedia's Chief Research Coordinator

This probably fits in a summary somewhere, but I'm just posting for anyone interested who couldn't make it. --MichaelBillington 11:36, 12 March 2007 (UTC)

Main points to be decided

Thanks for the notes above. The main points (for me) that we need to decide are:

• Is Wikiversity for educational research, or all types of research?
• Should we go with the Review Board idea, or something akin to a "flagging and deletion" (ie VfD) process (or something incorporating both)?

We need principles in place for all Wikiversities (like WP has NPOV, for example) - however, we have been discussing if every project has to stick to exactly the same mechanisms for dealing with research. I think it's most practical if we distill these policies into key ideas that new projects can use without needing to translate all the English language pages. Other ideas about how to attract researchers to Wikiversity are great, but unnecessary to deal with for the moment in our common policies. Please help out - remember, the board meets this weekend. Thanks. Cormaggio 14:57, 12 March 2007 (UTC)

Review Board

These are my three suggestions for the removal of research that goes against the rules/guidelines:

1. A group of community-elected Referees (the Review Board) scrutinise the method and intent of every posted research project (complete or otherwise) and removes inappropriate research (or requests custodians to do so)
2. All members can flag up questionable research with a template (and category), which is then scrutinsed by Referees and removed if inappropriate
3. Members could flag up questionable research with a template, and then a vote takes place for support/oppose of deletion, which is then actioned by custodians

In all these scenarios, members can of course propose speedy deletion of obviously malicious or vandal projects. - Xenon (talk) 17:18, 12 March 2007 (UTC)

It is inevitable that some research projects will be run through the Wikiversity page deletion process. We need to make sure that there is an efficient disposal process that will deal with any attempts to post bogus research to Wikiversity. As currently written, the research policy pages might place too much emphasis on page deletion. I'd like to make sure that the research policies make the point that many "problems" that will arise in research projects can be corrected as part of the process through which Wikiversity participants will learn about how to do good research. --JWSchmidt 18:59, 12 March 2007 (UTC)

What if all research could be placed on the request for deletion-like page, but primary research (New data) must be first approved by the referees? Secondary research will then fall under the reliable sources policy. If you don't have reliable sources, it's primary research and must be approved first. Just an idea.--Rayc 19:52, 12 March 2007 (UTC)

In one way, deletion is a response appropriate only for vandalism, total mistakes or irrecoverable failures. If projects are assessed in proposal form (before being carried out) then modifications and recommendations can be made to keep it within our guidelines. However, many projects cannot be modified and must be removed - we can't allow fringe projects (eg Nazi propaganda) to stay, and the ethically-questionable projects break the rules and should be taken off or at least hidden beneath a disclaimer template.

As for the rules, many projects may be just hovering on the 'unacceptable' line. That is why we should have two lines - and anything ethically questionable should not be allowed, giving us a margin from the line that would get us in trouble if breached - a safety net of sorts.

I think the community should be able to make decisions on its own - that currently includes removing vandalism (or requesting its removal), but should be expanded to removing (or covering up) ethically questionable (or guideline-breaking) research projects. A Review Board should be notified in the occurrence of borderline or questionable research, and they can make an informed decision based on their knowledge of the Research Guidelines. Perhaps another name could be the Research Review Team, so as not to be confused with an IRB. - Xenon (talk) 23:14, 12 March 2007 (UTC)

I think John makes a good point above that we are trying to promote learning about doing good research, rather than simply deleting "bad" research - this principle should be constantly borne in mind. I think that permitting research by default and then flagging it for review (perhaps under a set of criteria and categories) is more practical than having to have research reviewed before it can be added. I also think that the review process should be open to anyone who is interested - thereby opening the possibilities for developing critical thinking. If we need some sort of board to rule on whether something is too unethical to be maintained on Wikiversity, then so be it - I would see this board as the final referent - which will only act where there is significant controversy. Also, specifying that research should be reviewed at the proposal stage will not work, because people will be presumably uploading the results of their research (ie papers), by which stage the proposal is outdated, or even irrelevant. Cormaggio 00:16, 13 March 2007 (UTC)

Another name for the "research review board" might be "research advisors group". The way Wikiversity:Review board/En is written now, the main purpose for "referees" is to advise about good research practices: "help the community verify the methods used in research projects". As currently proposed, the only suggested unique function for referees (that is, a function not shared with other members of the community) is described in the "Pre-approval of research projects" section. If we are going to allow researchers to do IRB-approved research that involves collecting data at Wikiversity (and I'm not sure if that will ever happen) then I think we need some kind of over-sight to make sure that such research does not violate Wikiversity policy or Wikimedia Foundation policy such as the privacy policy. If a situation ever did arise in which someone wanted to do IRB-approved research with a component of that research project at Wikiversity, the research review board might constructively function as a bridge between the external IRB that approves the research and the Wikimedia Foundation office. All the details for such research and collecting data from Wikiversity participants would have to be worked out in the event that such a research project is ever actually formulated. Alternatively, we could just say that any research needing IRB approval is excluded from Wikiversity. I'd like to find a way to leave the door open for such research. I think it will eventually become possible for bricks-and-mortar institutions to do some interesting and useful educational research at Wikiversity. Such research would be facilitated by Wikiversity having some "official" point of contact for the IRBs of those institutions. --JWSchmidt 00:50, 13 March 2007 (UTC)

I think the Review Board (or Research Review Team, or Research Advisory Group etc) should examine every submitted project, as they would be the only members qualified (internally) to approve projects. However, projects should always be shown until the RB/RRT/RAGs assess it - thus it is not a system of approval/rejection before granting access, but a simpler system of acting upon projects at multiple stages of their development. The RRT would have access to a selection of templates which (only?) they should put at the top of each project.

For example: "This project has been approved by Wikiversity's Research Review Team" or "This project has been reviewed by...and has been deemed to be ethically questionable", and of course a few more ranging from full approval to please-change-this to straightforward unacceptability. These templates serve two functions - firstly they inform readers of the validity of the methods (and thus data), instilling caution where it's due; and secondly they advise the creators of the project on how to correct any problems. The RRT could also advise potential researchers of the best options to take when they are planning their projects. The RRT is not simply for deletion of the totally unacceptable projects, but an advisory council on the strengths and weaknesses of every project, at every stage. - Xenon (talk) 10:31, 13 March 2007 (UTC)

• Xenon, I am not sure what you mean by submitting a project, and examining a project. Does it take place before, during or after the work? Is it an informal check or a formal review? As I can see, the current Review Board proposal is also a means of recognising expertises in wikiversians and using them to guide the study projects in wikiversity. It may be somewhat daunting to put too much burdens and responsibilities on referees. It may be enough just to review those research projects which are more controversial, and those which are to be published.---Hillgentleman|書 12:04, 13 March 2007 (UTC)

• • The goal of any wiki is to be able to quickly notice problems on wiki pages and then fix those problems as quickly as possible. In my experience, there are some wiki "jobs" or "tasks" that tend to be boring and not accomplished very quickly by the community. In general, forcing people to do something (like review pages on a set schedule) is hard to do with wiki volunteers. If a backlog builds up for a task, you just have to try to get more people involved. In this case, I think we need to find ways to involve people with research experience in the Wikiversity project. We could think about a way to have stable versions or "peer reviewed versions", but in general there is just a continual process of people noticing problems on wiki pages and fixing them. I like the idea of having a required template to place on research pages in order to mark the page as being a research page. Maybe such a template could include a system where people could sign and indicate that they had reviewed the page and had either found reasonable research methods or noted problems on the talk page. --JWSchmidt 15:45, 13 March 2007 (UTC)

Hillgentleman - I was implying that the RRT would read and scrutinize every research project as soon as it appears or is significant edited, this could happen multiple times to a project through its lifecycle. However, the compromise is allowing normal editors to read and edit the projects, with the RRT only assessing the research that the community have marked as ethically questionable (eg: maybe breaking the rules).

1. every research project has to be clearly marked as such, with appropriate templatage
2. the community notices and corrects (or requests the correction of) problems, changing the template as necessary
3. Anybody who reads a research project can sign the template to acknowledge their belief that the research is okay
4. The RRT (Research Review Team) examine any research marked as 'questionable' by the community and corrects any issues
5. The RRT is a formal point of advice for anybody considering undertaking a research project, and they can advise on different methods/samples etc before the research is undertaken

So our goal is to get as much research posted, and the community can examine and edit all the projects all the time, just as they can with every other wiki page - but the community has the added responsibility of templating the research, or at least adding their signature of approval. This reduces the job of the RRT to just examining questionable research. Perhaps they would also have the casting vote in a discussion that wasn't sure whether or not to remove or modify part of a project? - Xenon (talk) 17:06, 13 March 2007 (UTC)

I think there's a common view emerging - Xenon's breakdown above is clear and manageable (my main concern has been echoing John that if all research had to be reviewed, we could easily generate a backlog, and backlogs invariably make people feel burdened and demotivated). So, with a distributed community process in place, and a research board/review team/advisory group to give advice (and arbitrate, and even interface with other institutions, where necessary), I think we easily have enough structure in place to deal with research. Of course, finer details will, as always, have to be worked out through experience, but that's all part of the learning process. Regarding templates, I like the idea to incorporate the review process within them (though is it user-friendly enough?) - in fact, I've always hoped that we could have a system in place where people can review any materials on Wikiversity, not just research. I think it's a good solid idea to flag research as research for a start - though does this need to be broken down into subcategories like "research report", "research-in-progress", research data", etc.? I'm going to start editing the pages now - please review... :-) Cormaggio 09:13, 15 March 2007 (UTC)

Educational research

We've addressed one of the major points that I outlined - but what about the other (i.e. are we limited to "educational research"?)? On Wikiversity:Research_guidelines/En it says: "The scope of research at Wikiversity is limited to research activities that promote learning and the goals of the Wikimedia Foundation." This suggests that we would be limiting ourselves to research which is specifically about promoting learning. What I want to raise is whether this is what we want. Cormaggio 22:10, 15 March 2007 (UTC)

• What does educational mean? Really, every piece of research is an act of learning: about the World, about ourselves, or about the relations between ourselves and the world..---Hillgentleman|書
• It reads like a self-referential definition, which I think is a little vague. However, there might be an advantage for it to be open-ended. --HappyCamper 16:45, 16 March 2007 (UTC)

Two very incisive comments. I think it's difficult to say what exactly is "educational research" - and, of course, all research can be of educational value, and be an educational experience. This question only came up during the IRC meeting and it kinda threw me - I had never personally considered research to be limited by topic before this. Right now, we are only specifically excluding "marketing", "illegal" or "unethical" research - the question for me is how do we exclude research on the grounds that it is "not educational"? Which addresses HappyCamper's point to some extent - how do we work with the definition as it stands? Cormaggio 17:46, 17 March 2007 (UTC)

• What is bad about allowing marketing research on Wikiversity? Is it because the information is useful only to one group of people? Or is it because it may not be objective?--Hillgentleman|書 21:27, 17 March 2007 (UTC)

The full item reads, "'marketing' research that promotes a specific commercial product or political candidate or any other kind of research that has as its goal something outside of the educational mission of Wikiversity" <-- I put "marketing" in quotes because I did not know of a good word to to cover all of the subcategories including political and marketing research. I had come across descriptions of marketing research such as, "Surveys of the area in which a product or service is to be offered, which are done to determine the cost of doing business, any competition, potential sales, etc." I had an image in my mind of some company making use of Wikiversity server resources so that they could conduct marketing research for a product. I think the key part of the wording in the policy is, "has as its goal something outside of the educational mission of Wikiversity". I someone wanted to have a project that would help people learn about how to do marketing research in general, that would be fine. My thinking was similar for other areas of research such as political research. For example, I do not think it would be good to allow partisan political research at Wikiversity that is designed to find "dirt" about candidates. When I wrote that part of the policy I had just heard about political campaigns spending hundreds of thousands of dollars on researchers who dig up dirt on political candidates. I like the idea that in our wiki projects we should, "leave our political fights at the door to the greatest extent that we can". I had an image in my mind of someone starting a partisan political research project using Wikiversity resources to host webpages that invite people to share dirt and negative information about political candidates. On the other hand, if someone wanted to do education-oriented political research that showed people how to do non-partisan political research, that would be great. --JWSchmidt 00:21, 18 March 2007 (UTC)

• Great. Sharing knowledge for knowledge's sake, not for personal advancement.---Hillgentleman|書 10:01, 19 March 2007 (UTC)

See also

English language discussions of research policy

Research guidelines Discuss scope of research - Research source documents - Original research - Secondary research - Discuss research guidelines - Discuss review board - Xenon's talk page - IRC meeting 10 March 2007