AI Research Tools: My Hands-On Experience with Elicit
Unveiling the real deal: What a new AI research assistant Elicit can and can't do
We've all seen those exciting headlines: "AI can do your research in minutes!" and "Let AI write your literature review while you sleep!" These promises sound amazing, especially when you're facing a mountain of articles to read through. But how well do these AI research tools actually work in real life?
I decided to put one popular tool, AI research assistant Elicit, to the test with a real problem I'm facing: my four-year-old daughter's extremely selective eating habits.
The Grand AI Research Experiment
This wasn't about testing something abstract. My daughter's eating situation is a daily challenge in our household. This isn't your standard "I hate broccoli" situation—it's a complex issue that emerged around age two, and I've spent almost three years trying to understand the underlying causes.
This made it a perfect test case since I already knew enough to judge whether the AI's research would be helpful.
Enter Elicit: The Research Assistant AI
Right away, this AI research tool impressed me with something unexpected. Instead of just running with my vague question, it asked which specific aspect of selective eating I wanted to focus on:
-Intervention potential
- Developmental perspective
- Comparative approach
This was genuinely helpful! As someone who's done plenty of research, I know how important it is to narrow down your question before diving in.
After I selected my focus, this Elicit reformulated my question. I could either accept this new version or edit it further to get exactly what I wanted.
While the AI worked, it showed cute coffee cup animations and progress updates. It reminded me of those robotic barista Tao Bin, famous in Thailand, that visually shows each step of making your drink - a small but nice touch!
What AI Tool Elicit Did Well
Finding real articles: Everything Elicit cited actually exists - no made-up sources! This is huge because some AI tools have been known to "hallucinate" research that doesn't exist.
Helping refine my question: The guided process helped me clarify exactly what I was looking for.
Providing a starting point: When you're tackling a new subject, sometimes the hardest part is knowing where to begin. This AI research assistant gave me that starting point.
Professional formatting: The final report looked polished and well-organized.
The Reality Check
Here's where expectations met reality. Elicit (its free version) promised to analyze 50 articles, but ultimately only included 10 in its report. And here's the bigger issue - of those 10 articles, it only accessed the complete text of 4 in my first search and just 1 in my second.
This isn't entirely the Elicit’s fault - it's a limitation of academic publishing, where many articles are behind paywalls. But it means this research assistant was mostly working with abstracts (those brief 150-word summaries) rather than complete papers.
This is like trying to understand an entire movie by reading only the blurb on the DVD case.
One more thing, the report looked well-organized but contained very general conclusions like "Food neophobia was common in children" or "Food refusal was associated with 4 factors" without specifying which factors. Important details I knew existed in the literature - like connections to micronutrient deficiencies including zinc, vitamin D, and folic acid - were completely missing.
How to Actually Use AI Research Tools Effectively
Despite these limitations, I found Elicit to be a worthwhile AI tool when used with the right expectations. Here's how to make the most of it:
Be super specific in your queries: Instead of asking about "selective eating in children," specify details like age range, whether the child is neurotypical, and any other relevant factors.
Use a step-by-step approach: Start with a broader question, then use what you learn to ask more focused follow-up questions. It's like building a pyramid of knowledge, one level at a time. For instance:
Level 1: "What causes selective eating in children?"
Level 2: "What role do micronutrients play in selective eating?"
Level 3: "How does zinc supplementation affect selective eating behaviors?Value the sources more than the summary: The most helpful part of my experience wasn't Elicit's analysis but the relevant articles it found. I still needed to read them myself to get the real insights.
Don't expect a complete literature review: Think of these tools as research assistants that find relevant sources, not as replacements for critical reading and analysis.
Who Might Benefit from AI Research Assistant “Elicit”?
Beginners exploring a new topic will appreciate the guided approach and overview of the subject.
Students might find it helpful for discovering relevant sources, formulating a solid research question and clearly defining their methodological approach.
Bloggers and journalists can use it to quickly find credible sources, saving time on initial searches.
For more advanced researchers like PhD students, the free version of this AI tool is less helpful since it primarily works with abstracts rather than full text. If you're already familiar with academic search tools like JSTOR (which offers free access to 100 articles monthly), you might not find this AI research assistant saves much additional time.
Keep in mind I’ve only tested the free version of this AI research assistant so far. The premium version probably performs much better. I’ll check it out later this week and let you know what I find.
The Big Picture: AI as Augmentation, Not Replacement
AI research tools aren't yet the magical "press a button and get complete research" solution that some headlines promise. But they can be vary valuable aids in the research process when used appropriately.
The key is understanding what today's AI can and can't do. Current tools are great at finding and organizing information but still struggle with the deep analysis and synthesis that makes research valuable.
Used correctly, tools like Elicit can save you time on initial searches and help you discover relevant sources. Just remember that you're still the researcher - the AI is just your assistant.
Have you tried Elicit or other AI research tools? I'd love to hear about your experiences in the comments!
*This week later: I'll be reviewing Scite, another AI research tool that takes a different approach to literature searches.*
This was solid. Really appreciated this. You're one of the few people writing about these tools without sliding into hype or fear. I’ve used AI for research too. Sociological analysis, long-form, productivity and I’ve landed in the same place: it’s useful, but only if you know what it’s actually doing.
The hallucination issue doesn’t go away. It never will. It just changes shape depending on whether you're giving it real data or asking it to go find some. Deterministic inputs - summaries, formatting, rephrasing known content, are cleaner. But once you step into speculative tasks, you’re back in probabilistic mode. More insight but more prone to hallucination. Could you restrict AI to only objective tasks? Sure. But then, you might as well just use a spreadsheet. The real power in the AI tool is accepting but also mitigating the risk when speculating. And not using output as conclusion.
So yes, AI tools are assistants, not oracles. They can accelerate the stages, gathering, organizing, framing but the thinking? Still yours.
Appreciated how you framed it: assistant, not answer key.
I think they can be like a really quick research analyst. Sometimes you will get mad but they can learn. And they don’t expect days off!