Artificial Analysis - Prolific Researcher Reviews

study 6acd969eeb54ba891194076f This is a complete scam. "By agreeing to these terms, you give your specific, informed, written consent for us and our licensees (including the developers of the AI models evaluated) to use your recordings to train, fine-tune, test and evaluate AI models, including speech and voice models. This may include creating synthetic, computer-generated speech or voice models that resemble your voice (a "digital voice replica"), which could say things, in any language, that you never actually said, and which may be difficult to distinguish from your real speech." "You grant us a worldwide, royalty-free, perpetual, irrevocable (to the maximum extent permitted by law), sublicensable and transferable license to use, reproduce, store, modify, analyse, create derivative works from and share the recordings and other study data you provide, including to train AI models and create digital voice replicas as described in Section 6, subject to the protections in Section 6. Where applicable law treats any right in your voice as personal and non-assignable, this operates as a license of that right to the same extent." They will literally own your voice and can profit from it forever and always. All for the low, low price of $10?

mic needed, 8 short ai conversations

You've got to watch these guys. Instead of the usual 50 prompts, this one had 75. While they claimed: This task consists of 75 prompts with a pair of videos for each. You will be tasked with voting for which video is a higher quality generation than the other. Each prompt takes about 20 seconds to evaluate. They are full of it. It took me almost 35 minutes because, guess what? 25 more prompts are going to take longer because the videos are about the same length as those in the 50-prompt studies.

great studies pay within few hours

This is the second study today from a researcher (albeit a different one) where I got into the study, completed the preliminaries such as consent form, reading the instructions, and mic and audio connection, but I couldn't proceed to the active part of the study because of an error message to the effect that there are no places available and to come back later. The error message for this one was, "Could not start - 429: {"detail":"The study is at capacity right now; please try again later."}". I'm not sure if something is out-of-sync with Prolific and researchers' servers today, but it seems to me that when you are able to accept a task, that means there *are* places available. Disappointing and a waste of my time.

Video Output Evaluation study, $4 for 30 minutes is far from fair remuneration.

This study appeared in a sea of studies by my favourite AI researcher and I clicked on it by mistake thinking it was one of theirs. Returned it after giving it a good go because it is an endless timesuck. They want you to do a quality review of 50 video prompts in 30 mins for $4! When there are 3 x 10sec videos to watch for each prompt. hahaha. Have blocked this researcher to make sure I don't click on one of theirs by mistake again.

Fairly simple to do study. You simply have a short conversation (~2-3 minutes) with one AI model, then do it again with a different AI model, and then rate which you preferred on things like instruction following, tone, etc. You then repeat for 4 total trials of that.

Quite a long study to do correctly as its by number of videos (They state 15 seconds per video, but each consists of 2, 10 second snips), rather than time taken, so it can (WILL) take longer than stated. There is no middle ground, forcing you to like one video or the other, as some are both good or both bad this can be difficult to select the 'right' one. Not sure that they obtain good results this way as some will simply rush.

Absolute insanity. The Mandarin Audio Preference Evaluation I was given had 2500 audio samples for me to evaluate - each one taking AT LEAST 6 seconds to listen to. And given the allotted 37 minutes, I do not see or understand how anyone could complete this at all.