Javi: Empowering Smarter Identity Management with AI and ML

Data Protection

What user data will Javi collect, process, or store during an access request?

In Omada Identity Cloud, an Access Request is always associated with an identity. The user account in Teams initiating the request is already linked to the corresponding Identity Record. While this information is necessary for processing and executing the access request, it is only retained in Omada Identity Cloud not Javi for audit and governance purposes.

As long as the Javi/Teams interface is in use Javi will continue to store this information in memory. The user prompts are not stored anywhere in the Omada system, however they are retained during the lifetime of the session. After 10 minutes of inactivity, the conversational history is deleted. Microsoft log the number of tokens used but not the user prompt.

To which data in Omada Identity will Javi have access to, and what are the limits of information that a user can retrieve via Javi?

Javi has access to the Identity and User information within Omada for the customers environment, depending upon the skill Javi will have access to Omada data from the Omada Graph API. Contact Omada for specific information on object/attribute mapping if required.

What type of user interactions or inputs does Javi process and retain? Which user activities or data points are logged, and for what duration are they stored?

Javi utilizes short-term memory, enabling it to retain and use information throughout a conversation, similar to how a human would. However, Javi does not have long-term memory and cannot remember details from past conversations. Its memory is limited to the active session or up to 20 prompts by default.

How is Personal Identifiable Information (PII) handled, and is it encrypted at rest and in transit?

PII is encrypted over the wire using TLS and any interrogation of the Omada Graph API is performed over TLS and Microsoft EntraID also perform encryption of PII.

Is data collection and retention compliant with GDPR, CCPA, or other relevant regulations?

Omada are compliant with GDPR, other relevant regulations include the EU AI Act.

Are session access logs to Javi anonymized or pseudonymized to protect user identities?

Details of session prompts are not logged. What is logged are the number of sessions and token usage within the session.

Consent and Transparency

How are users informed about the data Javi collects and processes?

Omada product documentation. @Nicole to follow up after thinking

Does Javi provide clear consent options for end-users?

When installing Javi in Microsoft Teams, users are presented with the Privacy Policy, Terms of Use, and Permissions, which define how Javi operates and interacts with their data. As a Teams app, Javi obtains user consent through this installation process. Any necessary updates to these terms to reflect Javi’s specific functionality will be incorporated as needed.

Is there a mechanism for users to request data deletion or review their stored data?

Javi does not store any user interaction data.

Authentication and Authorization

How does a user authenticate to Javi?

Javi authenticates users through their Microsoft Entra account directly within Microsoft Teams. As a Teams app, Javi utilizes implicit authentication by acting on behalf of the authenticated user. (Please refer to: How does Javi prevent privilege escalation or impersonation attacks?)

How does Javi authenticate itself during an access request?

Authentication is made using impersonation directly to the Omada Identity Cloud GraphAPI.

Is Javi integrated with multi-factor authentication (MFA) mechanisms?

Javi’s integration with multi-factor authentication (MFA) depends on the security settings configured by the customer for user logins in Microsoft Teams. Its implementation is determined by the customer’s chosen security policies within their Teams environment.

Threat Modeling

What are the potential attack vectors for Javi, and how have they been mitigated?

Omada has engaged independent third-party penetration testers to evaluate the security of the Javi solution. The outcome of these assessments was successful, with no actual attack vectors detected. Any perceived vulnerabilities were attributed to hallucinations rather than genuine threats. This rigorous testing process ensures that Javi remains secure against potential attack vectors.

How does Javi prevent privilege escalation or impersonation attacks?

Impersonation is a mechanism where the system acts on behalf of a user based on their credentials and permissions. This approach ensures that all actions occur within the user’s own permission scope, thereby minimizing unauthorized access. By preventing direct system-level privileges, impersonation reduces the risk of privilege escalation. Consequently, any user session is limited to the actions that the user is authorized to perform, enhancing overall security.

Auditing and Logging

What level of detail is captured in audit logs for access requests processed by Javi?

Audit logs for access requests processed by Javi include detailed labeling for all approvals performed. This ensures a comprehensive record of all actions taken within the system.

Are all conversations with Javi logged (or just the outcome in terms of access requests, approvals, etc.)?

Conversations with Javi are temporarily stored in the customer database while the user is actively interacting with Javi to support scalability. These conversations are deleted after 10 minutes of inactivity. However, the outcomes of these interactions, such as access requests and approvals, are permanently logged in Omada for auditing purposes.

How are logs protected against tampering?

Logs are stored in a separate, secure storage environment where editing is not permitted. This measure ensures the integrity and protection of the logs against any tampering attempts.

Integration with Existing instance of Omada Identity CloudIAM Systems

Does Javi inherit existing IAM security policies from Omada Identity Cloud?

Yes, the data retrieved via the Omada Graph API is scoped to the Users permissions that are defined by user groups within Omada’s security model.

How are failed access requests or anomalies escalated for human review?

Two definitions of a failed request.

(1) In the process of making an access request an error is returned from the Omada Graph API.

(2) Javi recognizes that you asked for a request, but the request was abandoned because the request prerequisites were not fulfilled.

AI Security Risks

How is the AI model protected from adversarial attacks or poisoning?

Javi is protected against adversarial attacks through a combination of prompt engineering, semantic kernel techniques, and robust coding practices. Prompt engineering involves carefully designing input prompts to minimize vulnerabilities and ensure accurate outputs. Semantic kernel techniques enhance the model’s understanding and processing of data, reducing the risk of manipulation. Additionally, rigorous coding practices ensure the integrity and security of the AI model, making it resilient against adversarial interventions.

Regarding data poisoning, it is not possible with the large language models (LLMs) used by Javi, as these models are not being trained on new data. This approach ensures that the integrity of the model is maintained and prevents any malicious data from influencing its performance.

What safeguards prevent Javi from inadvertently leaking sensitive information?

Javi prevents the inadvertent leakage of sensitive information by leveraging Graph API and data object security. This ensures that Javi can only perform actions and access information that is available to the authenticated user. Furthermore, independent penetration testing of the Javi system has confirmed that the controls and architecture in place effectively prevent the leakage of sensitive information.

Model

Does each customer get their own LLM model?

No.

What are the limits of what you should ask Javi?

Javi’s skills at point of release are related to the following IGA skills:

• Make an Access Request
• Approve Access Requests
• Set Notifications for reviewing your pending requests
• Access the Omada documentation

Any guidance on how to talk to Javi?

Always state your intent at the beginning. Avoid adding any further details at first. ”I want to request access”, ”Show me my pending approvals” should be sufficient. Javi will prompt you further.

Avoid ambiguities. For example, you have just requested to see your pending approvals. You then ask to review further the most recent 5 from the list. Finally, you say ”Give me more info on the first 3″. Javi will be confused between the first 3 resources from the first overview and the second one.

Maintain relevance to the most recent turn. For example, you are requesting access to a resource. Javi has gathered your business context and selected resources and is now asking you for the reason. Instead, you respond “I want access to resource [a second resource] to carry out a project”. Javi may be confused over which action to perform next.

Avoid abrupt shifts of scope. For example, you are in the process of making an access request. You have asked Javi to help you select a business context for your request. If you ask Javi at that point, ”What are the approval requests pending to that context”?, he will most likely be confused.

Use the keywords “search” or “find” if Javi is having trouble retrieving information, such as your pending approvals, resources for requesting access, resource ids for requesting access, your business contexts etc.

Paraphrase your prompt, when Javi is not returning the desired response.

Be conscious of memory capacity when requesting access. Avoid requesting access to more than 2 (possibly 3) resources per request. Maintain a step-by-step and fast conversation flow when filling in the requirements for the access request (1. Select context, 2. Find resource_A, 3. Find resource_B, 4. State reason, 5. Confirm Summary, 6. Make request).

Avoid overloading your prompt. Your prompt should focus on performing a single step. For example, if you want to request access to multiple resources during an access request, search for them one by one using one turn per resource search.

User Experience

How intuitive is the interface for initiating and tracking access requests via Javi?

Initiating and submitted access requests with Javi was designed to be simple for end users with their own natural language interactions.

Does Javi provide feedback or guidance when it cannot fulfill a request?

Yes, superficially, Javi provides guidance advice to seek other advice and where to go for that advice in case a request cannot be fulfilled such as documentation assistance and persona (administrator) assistance.

Is Javi alerting the end user (e.g. when he/she has to do an approval)?

Yes, this is part of the notification’s skill

Integration

Which API is Javi using for requesting access in Omada Identity?

Omada Graph API

What is, in a nutshell, the architecture of the Javi component?

Omada Graph API is used to access data and interact with workflow (business process).

Depending upon the skill being used Javi will use plugins within the Semantic Kernel to perform the necessary action on the relevant API such as the Omada Graph API.

Javi lives in Microsoft Entra as a OpenAI LLM with configured skills that are orchestrated by the semantic kernel to achieve the goals of the user prompt.

How is Javi activated and configured? Which configuration options are there?

Javi is activated by the user from the Microsoft store, there is no configuration required by the end user.

What systems and applications does Javi support for access requests?

Users can request access to any system or application that is within their business context, there is no change in availability of systems or resources from the web experience.

Are there APIs or SDKs for seamless integration with third-party tools?

No.

Can we develop custom skills and let Javi deal with them?

No.

Performance and Scalability

How does Javi handle high volumes of simultaneous requests?

Javi is equipped with a model load balancing protocol that is specifically configured to manage high volumes of simultaneous requests. This ensures that the system remains responsive and efficient, even under heavy load conditions.

What is the response time for processing typical access requests?

Processing typical access requests with Javi involves multiple prompts to gather and confirm the necessary information. Despite this, users experience should not experience any delay and no noticeable difference in the user interface compared to the current solution.

Error Handling

How does Javi handle incomplete or ambiguous access requests?

Javi will prompt the user to complete their request by suggesting elements of the request that are required such as the business context, the reason, the requested resource. Javi will provide information and make suggestions such as fuzzy string matching and cross language assistance, spelling errors etc.

Javi has access to resource descriptions that Javi uses to help provide the user select the resource to request.

Javi also has access to the Omada product documentation that can be used to provide context to the request.

What fallback mechanisms are in place if Javi encounters a failure?

Depending upon the Javi skill in use when the failure occurs, the fallback mechanism could be to revert to the Omada web portal.

Policy Alignment

How does Javi ensure access requests align with the organization’s access control policies?

Yes, Javi creates requests through the Omada Graph API which follows the control policies set in Omada for the user session.

Does Javi take SoD constraints and other risks into account?

Javi ensures that Segregation of Duties (SoD) violations are visible to the approver but, by design, are not disclosed to the requester. The approver has visibility of any policy violation, prior to the new access becoming active the violation needs to be addressed by the designated user which can also be the approver. The discretion to determine how to handle the policy violation during the approval process.

Auditability

Are all Javi-generated actions auditable for internal and external compliance checks?

Post GA a priority roadmap task is to assign a label to the approval audit records within Omada to mark them as assisted by Javi

When access requests are initiated through Javi, how is accountability ensured so that each request can be reliably traced back to the authenticated user who initiated it?

Access Requests performed by Javi flow through the Omada Graph API which is used by other external components (such as the ServiceNow integration and others), there is nothing specific relating to auditability for Javi when access requests are made. Access requests via Javi are performed by the impersonated user, using Javi as the client. Javi leverages the Graph API, ensuring users act within their permissions. Each request is recorded for auditability, tracing back to the authenticated user.

Third-Party Involvement

Are any third-party services used for Javi’s operations? If so, are they compliant with privacy and security standards?

Microsoft Teams and yes Microsoft are compliant with relevant privacy and security standards.

Monitoring

How is Javi’s behavior monitored to ensure it performs as intended?

Javi’s behavior is meticulously monitored through a comprehensive evaluation process that is implemented prior to the deployment of any new large language model (LLM). This pre-deployment evaluation involves rigorous testing to ensure that the model meets the required performance standards and behaves as intended.

In addition to these initial evaluations, future plans include incorporating user feedback mechanisms to further refine Javi’s performance. By gathering user evaluations, we aim to understand whether the responses provided by Javi align with user expectations. This continuous feedback loop will help us identify areas for improvement and ensure that Javi consistently delivers accurate and satisfactory responses.

Are there mechanisms for detecting and mitigating misuse or unusual behavior?

Omada employs robust mechanisms to detect and mitigate misuse or unusual behavior within the Javi system. These mechanisms include anonymized tracking of LLM token usage and monitoring the count of completed actions by skill, per user, and per customer. This anonymized tracking ensures that individual user identities are protected while still allowing for the identification of anomalies and outliers, which can indicate potential misuse or unusual behavior.

Additionally, the LLM is equipped with filters designed to flag and prevent the processing of harmful or malicious content. These filters ensure that any content deemed inappropriate or dangerous is promptly identified and blocked, maintaining the integrity and security of the system.

Updates and Patching

How are updates deployed to Javi, and are they tested for security vulnerabilities before release?

Yes, Javi regularly undergoes penetration testing by third parties.

Is there a rollback plan in case of an update failure?

Yes, versioning is in place in case of update failure.

Feature Expansion

What governance is in place to ensure future iterations of Javi (e.g., for more advanced IAM tasks) remain secure and compliant?

Omada are committed to maintaining regulatory compliance with the necessary specifications, for more information please refer to the Omada Trust Center available at omadaidentity.com or contact Omada support.

User Feedback

How will feedback from users and administrators be collected and acted upon to improve Javi?

It is planned for Javi to have a feedback skill, this is a roadmap item. There are also plans for an “up/down” voting feature to be exposed directly in the Javi Teams interface.

Adoption and Training

What resources or training are available for administrators and users to adopt Javi effectively?

Social media resources available to demonstrate skills, Javi will also have a handbook available as part of the overall product documentation.

How can a prospect try out Javi?

Contact their Omada support representative.

Can customer be part of a trial without purchasing commitment?

Yes, customers can trial Javi.