✨ Impact Statement
We reimagined the S Pen as an intelligent productivity tool—blending voice, gesture, and sketch into one expressive system. Our work laid the foundation for Samsung’s AI-powered stylus experience, shaping cross-device strategy and influencing flagship product launches.
🌟 My Role
As the Product Design & Interaction Design Lead on a 4-person innovation team, I led the vision, design strategy, and core interaction models. I collaborated closely with a junior researcher, a computational engineer, and a product lead—guiding early concept definition through to high-fidelity prototyping.
🧩 The Challenge
How might we evolve the Samsung S Pen into a dynamic, intelligent companion for everyday productivity—without overwhelming users?
While the S Pen was known for drawing and handwriting, most users weren’t tapping into its full capabilities. With the rise of generative AI and Samsung’s expanding ecosystem, we saw a clear opportunity: design a smarter, assistive, and context-aware S Pen that remains lightweight, intuitive, and fast.
We were asked to envision what the S Pen should become in 1–2 years, leveraging:
Multi-device capabilities (Tab S11, S25, Book5)
AI assistance and personalization
Seamless integration within Samsung’s productivity apps
🧠 Insights & Research
To define the next-gen experience, we ran:
📋 220-person quantitative survey (usability, AI interest, pain points)
🗒️ 10-person diary study capturing real-world S Pen workflows
🤝 22-person co-creation workshops exploring ideal experiences
✅ 380-person survey validating proposed features
Key insights:
Too many manual steps in creative workflows
AI felt disconnected from everyday tasks
Device switching (Tab to PC) broke flow
People wanted assistive features—but only if they stayed in control
“I want the pen to do more—but I don’t want it to take over.”
💡The Vision
We envisioned a stylus that thinks with you:
Understands context
Works fluidly across devices
Supports creativity and precision
Reduces friction for both power users and everyday users
The new S Pen would combine sketch, gesture, and voice into one intuitive interaction model, anchored by three design pillars:
Pillar | Focus |
|---|---|
Control | Fast sketch, gesture, and voice-powered interactions |
Precision-Guided Creativity | AI-powered content enhancement, generation, and feedback |
Omni-Sync | Cross-device fluidity: Phone ⇄ Tab ⇄ PC |
💡 Opportunity Framework: Moments That Matter
We mapped opportunity areas across four productivity moments:
Moment | Opportunity |
|---|---|
Setting Up | Speed up formatting, layout, prep work |
Creating | Enable fast sketch → AI support (generate, improve, finish) |
Upskilling | Encourage writing/creating better through subtle guidance |
Cross-Syncing | Let users move across devices without friction |
✨ Signature Experience Moments
We crafted six high-impact moments that showcased how the AI-powered S Pen could simplify and elevate everyday productivity:
Scenario | Feature | Outcome |
|---|---|---|
🖊️ Sketching a slide layout | Creation Wizard | Instantly turns sketches into layout suggestions |
🎤 Speaking while circling text | Voice-to-Visual | AI suggests matching visuals and format upgrades |
👆 Drawing with gestures | Contextual Panel | Triggers smart options like summarize, rewrite, generate |
✍️ Handwriting a draft | Easy Edit | Cleans up, expands, or improves tone instantly |
📎 Highlighting tasks | Scribe Sync | Adds items to calendar or reminders with a tap and voice |
😄 Doodling a shape | AI Sticker Doodle | Transforms sketch into fun, expressive visuals |
🎨 Design Process
Mapping Intent Across Inputs
We explored how sketch, tap, voice, and air actions could serve as natural prompts. Each modality mapped to different task types (e.g. circle = select, sketch = suggest).
Prototyping Contextual Menus
We built flows for the Dynamic S Panel—a floating, context-aware menu triggered by gesture or voice. Trigger methods included:
Shape detection
Tap on selection
Press and hold with voice
Air Actions (e.g., swipe or hover)
User Testing & Validation
Through in-person testing and surveys:
Users preferred suggestion-based AI over auto-actions
Wanted control: clear undo, visual cues, and feedback
Appreciated nudge-based onboarding vs. tutorials
🔧 Key Features
🪄 Dynamic S Panel
Context-sensitive menu triggered by gesture, sketch, or voice. Key tools:
✏️ Rewrite
💡 Summarize
🎨 Generate visual
💬 Smart tone shift
✨ “Surprise Me” creative AI prompt
✍️ Prompt-Based AI
Use handwriting or sketch to trigger intelligent assistance:
Draw a chart outline → Suggests formats
Write “add bullet list” → Converts instantly
Circle sentence + say “Make it friendly” → Tone adjusted
🧠 Creation Wizard
Draw a frame, label it “Infographic” or “Pitch Slide”—the S Pen suggests layouts, visuals, and content via AI.
✂️ Easy Edit
Tap a sentence/image to expand, clean, reword, or auto-align
🎤 Scribe Sync by Voice
Speak while highlighting:
"Add this to calendar" while highlighting text → AI schedules task ( calendar reminder)
“Share to chat” → Opens share menu with preview
🤖 AI Sticker Doodle
Draw something quirky → AI turns it into a stylized sticker using Bixby Vision.
🌐 Cross-Device Flow
The S Pen experience was designed to seamless sync across:
📱 Galaxy Phone
📒 Galaxy Tab
💻 Galaxy Book
Seamless hand-off allows users to start sketching on Tab, edit on Phone, and finalize on PC—with synced content, notes, and suggested actions traveling with them.
✨ “Draw here. Speak there. Sync everywhere.”
S-PEN Vision Video
Released: Galaxy AI's features, Drawing Assist
Released: Galaxy AI's features, Notes Assist
⚖️ Ethical AI & Accessibility
We ensured the experience remained transparent, inclusive, and empowering:
🔒 Clear feedback when AI is active
🎛️ Control-first UX (every action is reversible or reviewable)
🧏 Sketch + voice input reduces reliance on tiny tap targets
🧠 Onboarding happens naturally: nudges, not long tutorials
📊 Impact & Validation
93% felt more productive using the AI-enhanced S Pen
80% said they’d use it more frequently
Top 5 most used features (from prioritization survey):
Auto Sync
Easy Edit
AI Sticker Doodle
Prompt-to-Visual
Rewrite with Tone
Top 5 easiest to integrate:
Smart Menu
Sync Across Devices
AI Visual Generation
Tone Shift
Voice Action + Prompt
🧠 What I Learned
✅ Users are ready for AI—but they want it on their terms
✅ The best UX blends voice, gesture, and sketch with soft automation
✅ Personalized productivity = letting people shape how they work
✅ Designing across form factors (Tab, Phone, PC) means thinking in flows, not screens
🏁 Outcome
We delivered a compelling vision for the AI-enhanced S Pen that influenced Samsung’s cross-device experience:
Unified gesture, sketch, and voice into one natural system
Designed tools that feel magical but grounded in utility
Shaped Galaxy's broader AI strategy
Built scalable UX for AI onboarding and control
We led early exploration in 2023, which influenced features seen in Galaxy Unpacked 2024 and Apple Intelligence (June 2024).
✨ A pen that doesn’t just respond—but understands.
This work helped shape Samsung’s strategic narrative around the S Pen as a next-gen productivity tool, not just a hardware accessory.
📦 SDIC Innovation in the Wild
The concepts we developed at SDIC weren’t just theoretical—they helped shape real product experiences and influenced broader industry trends.
✅ Seen in Samsung Galaxy Launches
Our work wasn’t just speculative—prototypes like Easy Edit, Draw to Generate, and Content Wizard helped shape:
Galaxy Watch 6
Galaxy Tab S9
Galaxy S24 (Unpacked July 2024)
👀 Reflected in Apple Intelligence (June 2024)
Samsung concepts like Math Note, Image Wand, and Genmoji show strong parallels to our early SDIC explorations—a testament to our industry influence.
📊 Summary Timeline
This timeline comparison highlights how our features—like Dynamic S Panel, AI Sticker Doodle, and Reliving Memories—have helped define Samsung’s point of view on AI productivity and visual communication.
🤝 Learn more
Want to shape your own intelligent tools? I’d love to help. Let connect!