OpenAI has rolled out a new feature called Work with Apps in its ChatGPT desktop app for macOS, allowing the chatbot to read code directly from popular developer tools like VS Code, Xcode, TextEdit, Terminal, and iTerm2. This update aims to simplify the coding process by eliminating the need for developers to copy and paste code into ChatGPT manually. When enabled, the feature automatically sends the relevant sections of code as context alongside user prompts.
Unlike established AI coding tools such as GitHub Copilot or Cursor, ChatGPT cannot yet write code directly into these applications. However, OpenAI sees Work with Apps as a foundational step toward creating more agent-like AI systems. The key challenge remains enabling AI to understand the broader computer environment, beyond simple text inputs and responses.
Initially focused on coding tools, this feature is available for Plus and Teams users, with plans to extend it to Enterprise and Edu tiers in the coming weeks. OpenAI also plans to expand the feature’s compatibility to other text-based applications, such as writing tools.
In a demonstration for TechCrunch, an OpenAI employee used the feature to analyze an Xcode project representing the solar system. After prompting ChatGPT to "add the missing planets," the chatbot suggested a line of code for Earth that fit seamlessly into the project. Users still need to copy the generated code manually back into their development environment.
The system leverages macOS’s accessibility API to read text from supported apps, translating it into ChatGPT’s input. According to Alexander Embiricos, OpenAI’s desktop product lead, this API—used for Apple’s VoiceOver—has been reliable for most apps but falls short in handling visual elements like images or videos.
For some applications, such as VS Code, Work with Apps requires a dedicated extension to query content. The feature sends up to 200 lines of recent code or the full content of the active window, depending on the app. Users can highlight specific sections to refine ChatGPT’s focus, but surrounding context is also included, potentially increasing token usage.
Expanding beyond Apple’s screen reader limitations poses a challenge. Anthropic, a competitor, is exploring AI agents that analyze desktop screenshots to interact with apps. However, this approach remains slower and less reliable. OpenAI emphasizes that Work with Apps is not an agent but a tool designed to enhance collaboration with existing coding software.
This development coincides with reports that OpenAI is working on a general-purpose AI agent, codenamed “Operator,” expected to launch in 2025. Meanwhile, Work with Apps is available exclusively on macOS, ahead of Apple’s planned ChatGPT integration in December. A Windows release has yet to be announced, despite OpenAI’s close ties with Microsoft.
