Artificial Analysis - Prolific Researcher Reviews

Completely ridiculous that we are expected to do 80 of these in 20 minutes when some of them have very long prompts we have to read. Absurd.

Estimated completion time was listed as 20 mins but it took me 40 mins to complete. The description said it should take a maximum of 10 seconds to evaluate each prompt, but it honestly takes much longer than that to make a good evaluation. If I took a maximum of 10 seconds then my evaluation would be sloppy and carelessly, unless they're looking for you to evaluate things at a glance.. (but honestly without taking a closer look it's hard to tell which one is better than the others a good portion of the time)

Audio study had constant technical issues and kept giving me "request timed out" and error messages, so it took me 10 minutes more to complete it than it should have. I noticed if you can't get it to work, if you go back to Prolific and re-open the study from there, it would usually start working again.

Grossly underestimates the time it would take to do the study. Claims that each prompt should take a maximum of 15 seconds to evaluate, but the videos in the first set exceeded that time limit, let alone accounting for reading the prompt itself. Requires 50 comparisons in 20 minutes. Not a reasonable time limit for the amount of work required, regardless of the pay (which is low). I returned after seeing the first prompt.

Unfortunately after trying in 3 different browsers without success, I had to return this one. I kept getting an error when submitting my first answer, and I attempted to message the researcher, but never got a response. I am not sure why the errors occurred, but I was bummed to have to miss out.

80 image questions are supposed to take only 12 minutes to complete? And 40 video questions are supposed to take 14 minutes? No way. Super underpaid and actual time is double if not triple the time advertised.

Nice and easy. Not an auto-approved, but I submitted around 6:00pm Friday, by 8:00am Saturday it was approved.

Intended completion was listed as 30 mins but it took me 25 mins to complete. The study was fun and easy to get through :> I honestly spent some extra listening to the clips.

Great study. No complaints. Wish it were all this easy.

Pays okay, feels like I have to rush a bit to get through in the intended time though.