Mastering Sources, Prompts, and LLMs
Building critical AI literacy through multilingual, cross-platform financial research
Time to Complete: 30 minutes
Who This Is For: Students in personal financial planning with introductory LLM experience
Context: Foreigners navigating Japan’s tax and insurance system using AI-assisted research
Core Skills: Prompt engineering • Cross-lingual research • Source evaluation • LLM comparison
Why This Workshop Exists
LLMs are not research engines. They are prediction machines trained on text — and what they retrieve, hallucinate, or miss depends heavily on how you ask, where they look, and what language you use. This workshop confronts those limitations directly.
By working in both English and Japanese, comparing multiple models and search engines, and toggling between linked sources and uploaded PDFs, you will develop a calibrated understanding of what LLMs can and cannot do when the stakes are real — in this case, your taxes and your insurance coverage in a foreign country.
Learning Goals
By the end of this workshop, you will be able to:
• Conduct independent multilingual research and evaluate sources before engaging their content
• Identify meaningful differences in LLM output depending on how information is provided (link vs. uploaded attachment)
• Recognize that original-language sources are often more authoritative, and that LLMs can unlock access to them
• Practice prompt engineering as a deliberate, high-leverage skill — not a shortcut, but a craft
Workshop Steps
Work through each step sequentially. Your responses to the reflection prompts embedded in each step form the basis of your class discussion.
01
Search Engine Analysis
Prompt Google, DuckDuckGo, and Bing with the following query: “How to navigate the tax system, including insurance, in Japan as a foreigner.” Assess the sources of each listing without clicking on them.
Reflection: What patterns do you notice across platforms? Which engine surfaces government or official sources? Which leans toward blogs and forums?
02
LLM Comparison
Enter the exact same query into three LLMs of your choice. Assess the sources cited or generated by each model without clicking through.
Reflection: How do the LLMs differ from each other and from the search engines? Are the citations traceable? Do any feel fabricated?
03
Critical Decision: Link or LLM?
Decide whether you will use a source from a search engine or one generated by an LLM. Commit to one and justify your reasoning.
Reflection: What criteria matter most to you — recency, authority, accessibility, or something else?
04
Cross-Lingual Verification
Translate your original query into Japanese and repeat Steps 1 and 2. Assess whether the results are the same or different, and whether the Japanese-language sources feel more authoritative.
Reflection: What does the shift in language reveal about how information is organized and indexed for different audiences?
05
Source Selection
Choose one Japanese-language source from your search. Justify your choice and share the URL with the class.
Reflection: What signals of credibility did you use? Domain type, author, date, institution?
06
Prompt Enhancement
Access an LLM of your choice and submit the following meta-prompt:
Meta-Prompt to Use
Enhance this prompt: I am a foreigner who is now in Japan and will be living in the country for the next four years. I need a thorough tax and insurance financial plan. I will provide you with a source to develop this plan. Generate this plan as an editable Excel document.
Review the enhanced prompt carefully. Edit it manually if needed — you are the author, not the LLM.
07
Execution & Review
Input your finalized prompt into the same or a different model. Include the Japanese source you selected in Step 5. Review the output and note what surprised you, what was missing, and what was genuinely useful.
Reflection: How did the inclusion of a source change the quality or specificity of the plan?
08
PDF Mode: Attachment vs. Link
Return to your Japanese-language search and add ‘PDF’ to the query. Assess whether the recommended PDFs are recent and reliable. Choose one, click the link, and download it.
Reflection: How does PDF availability signal different kinds of institutional credibility?
09
Output Analysis: Link vs. Attachment
Upload the PDF to the LLM you used in Step 7. Run the same enhanced financial planning prompt. Compare the output you received from the linked source versus the uploaded attachment.
Reflection: Which output do you prefer, and why? What does the difference reveal about how LLMs process information?
10
Self-Evaluation
Return to the learning goals at the top of this plan. For each goal, assess honestly whether the activities helped you achieve it — and if not, why not.
Reflection: What would you do differently next time? What questions does this workshop leave open?
What to Look For Across the Workshop
As you work, pay attention to the following patterns:
SOURCE
Not all sources are equal — and LLMs do not always know the difference. A government PDF from the National Tax Agency is not equivalent to a personal finance blog, even if both appear in the same result set.
LANGUAGE
The language of a query is not neutral. Japanese-language searches often surface primary institutional sources that English searches miss entirely. This is a structural feature of how the web is indexed, not a quirk.
PROMPT
A vague prompt produces a generic response. The quality of LLM output is a direct function of the care you put into the prompt. Prompt engineering is not a workaround — it is the work.
FORMAT
Attaching a PDF gives the LLM direct access to the text. Providing a link does not — most models cannot browse the web unless explicitly equipped to do so. The difference in output quality is often dramatic.
A Note on AI and Financial Advice
Nothing generated by an LLM in this workshop constitutes legal or financial advice. The goal is to develop a research scaffold — a structured way of gathering, verifying, and synthesizing information — that you could bring to a certified tax professional or financial advisor.
LLMs are powerful amplifiers of research effort. They are not substitutes for professional judgment, and they are not infallible. The critical thinking you practice here is the skill that makes the technology useful.
#AILiteracy #PromptEngineering #CrossLingualResearch #PersonalFinanceJapan #LLMComparison