From speech to discoverable information – AI helps record and make use of ideas

A multimodal AI application combines speech recognition and advanced language models into an intelligent solution for note-taking and information management. Users can capture content by speaking or typing and supplement their notes with various attachments.

The project "Efficiency, Productivity, and Competitiveness through Data" has demonstrated and piloted the potential of Large Language Models (LLMs) by developing such an application. The experiment combined Microsoft Speech Service for speech recognition with OpenAI language models and investigated how AI can support the capture, processing, and subsequent utilization of information.

Speech is first converted into text, after which the language model corrects recognition errors and improves the text's flow. These corrections leverage domain-specific vocabulary, previous notes, attachments, and user-made edits. Additionally, the application generates summaries and automatically creates keywords for search purposes. Users can edit the content, adjust the correction sensitivity, and review the original speech recognition output.

There are also effective paid mobile applications available on the market. However, their suitability for corporate use depends on factors such as data security, information management, and customizability. Data processing and security are assessed based on the service provider's documentation and agreements; furthermore, features such as SharePoint integrations and the development of company-specific functions depend on the capabilities offered by the service. In the application being developed for the organization, data flow, access rights, integrations, and features supporting specific workflows can be designed to meet the organization's requirements.

Beyond note-taking, the functionalities implemented in the pilot can be applied to use cases such as:

• AI-assisted organizational feedback channel: Feedback from staff or customers can be collected via speech and text, categorized by topic, and compiled into summaries that support development efforts.
• Maintenance and field work reporting: Observations recorded verbally during work tasks can be processed into report drafts and linked to data concerning the specific site or asset.
• Capturing expert knowledge: Employee experiences, best practices, and work instructions can be recorded and structured into information that is easily retrievable and usable later.

The initial pilot version of the application focuses on desktop use. Based on feedback, mobile usage is a natural direction for further development, as it would facilitate the recording of ideas and observations the moment they arise.