Naavi.org

The Article 11 of the EU AI act states that there shall be a “Technical Documentation” of high risk AI system before that system is placed on the market or put into service and shall be kept up to date.

This is a document that a “Deployer” should obtain from the developer or supplier of the AI algorithm as part of the compliance requirements.

Under DGPSI*, which considers the AI algorithm supplier as a Joint Data Fiduciary of the deploying company, the deployer needs to obtain an undertaking from the supplier a conformity statement as part of the Contract and also assume liability for any non compliance of DPDP 2023.

EU AI act prescribes the format for documentation which requires the following format of documentation (Annex IV) which is also relevant for DGPSI compliance.

1.A general description of the AI system (including the purpose of usage)

2.Detailed description of the elements of the AI system and the process for its development (Applicable to the developers and including documented test process)

3.Detailed information about monitoring, functioning and control of the AI system

4.description of the appropriateness of the performance metrics for the specific AI system

5.Description of the relevant changes made by the provider to the system through its lifecycle

6.List of harmonized standards applied in full or in part.#

7. A Copy of the EU declaration of Conformity##

8. A detailed description of the system in place to evaluate the AI system performance in the post market monitoring plan.###

#List of Union harmonization legislation as per Annex II includes GDPR and other industry regulations where AI may be used as part of the system. In the Indian context this includes the ITA 2000 and the AI advisory.

# #DPDPA Declaration of Compliance

###In EU AI act, providers need to establish and document a post-market monitoring system in a manner that is proportionate to the nature of the artificial intelligence technologies and the risks of the high-risk AI system. (Ref Article 61).

US has called this “Process Controller” as Chief AI officer which is mandatory for federal agencies.

In the Indian context this is included in the AI policy managed by the DPO with the “Process Controller” under the distributed responsibility policy.

Under DGPSI the highlighted points are key to compliance with a modification that point no 5,6 ,and 7  should refer to DPDPA Compliance and point number 8 to the measures undertaken by the deploying Data Fiduciary. Points 2 and 3 are more relevant for compliance in the developer eco system.

*PS: DGPSI or Digital Governance and Protection Standard of India is the indigenous framework developed by FDPPI/Naavi for compliance of DPDPA along with ITA 2000 and BIS draft standard of Data Protection.

(…to be continued)

Naavi

EU AI act introduces a new terminology “Conformity Assessment” to mean a “Compliance Assessment”. In GDPR the term used was Privacy by Default. DGPSI uses a term “Compliance By default”. Now “Conformity Assessment” stands for an assurance certification of an AI system that indicates whether the requirements of Title III, Chapter 2 relating to the high risk AI system have been fulfilled.

Article 3(20) defines ‘conformity assessment’ as “the process of demonstrating whether the requirements set out in Title III, Chapter 2 of this Regulation relating to a high-risk AI system have been fulfilled”

This covers the following articles:

Article 8- Compliance with the requirements

Article 9: Risk Management System

Article 10: Data and Governance

Article 11: Technical Documentation

Article 12: Record-Keeping

Article 13: Transparency and provision of information to deployers

Article 14: Human oversight

Article 15: Accuracy, robustness and cyber security

Let us explore this further in the next article.

Naavi

It has been reported that in the Rameshwaram Cafe blast, one person who had bought a SIM card/second hand mobile from a shop was questioned since his number was involved in the communication related to the blast.

The seller of the mobile  has since been cleared and it has been identified that the SIM card buyer had misused the credentials of some other person to create fake ID and used it on the second hand mobile. A similar incident had occurred a few years back when a property owner in Bangalore had been falsely accused in a terror case because a fake Adhaar card had been issued in his name and used by the terrorist.

When such frauds occur, the dealer who created the fake ID becomes an accomplice and needs to be punished. At the same time, the telecom company which appointed the dealer is also liable for the same offence.

The offence comes under ITA 2000 under different sections such as Section 66,66C,66D,66F , Sec 43 etc. The same offence gets recognized under DPDPA 2023 as a failure of compliance for which penalties may be imposed. (When the act becomes fully operative).

In some of these cases, the telecom operator who may be Vodafone or Airtel or JIO etc provides two kinds of defences stating that it had followed “Reasonable Security Practices” under Section 43A and also that it could be considered as an “Intermediary” and protected from liability under Section 79 of ITA 2000.

In some cases the companies indicate that they had taken an ISO 27001 certificate which should be treated as a “Deemed Compliance of Section 43A”.

In this context, I would like to state my views why telecom companies need not be complacent that they have ISO 27001 certification and it can protect against being held liable under Section 43A or 43 or 85A and other sections of ITA 2000 both for civil penalties and criminal punishments of the executives .

In the ICICI Bank vs S Umashankar case, (para 8-15, page 9-16) the TDSAT in the appeal held that if security practices are not followed, Section 43(g) may be applied for “Facilitating the contravention through negligence” against the company (In that case it was the Bank but the principle is applicable for the telecom company for negligence in SIM Card issue).

Whether having an ISO 27001 certificate is an adequate security practice did not come for discussion in the Umashankar case. However in another case in TDSAT recently this discussion has come to the fore in the issue of SIM Card at the retail store/agent .

Since the ISO certificate was on a different system and a different date, it had no relation to the SIM Card issue process. At the same time since SIM card “Activation” is done only by the authorized official, the retail store agent is only a contractor to verify the KYC documents and recommend activation. Hence the telecom company cannot claim the “Intermediary” status. Also, the KYC information is not an “Intermediary’s data” but is the “Data of the telecom operator for its own consumption” and hence cannot provide the intermediary status to the telecom company under Section 2(1)(w) of ITA 2000.

It is further stated that from 1st December 2023, introduced new rules stating that all customers applying for new or replacement of SIM has to go through the KYC process.

“The guidelines also state that all telecom operators are now required to register their franchises, PoS agents, and distributors. Further, they will have to undergo verification. Failure to comply will result in a fine of Rs 10 lakhs. Point-of-Sale (PoS) agents must register themselves through a written agreement with licensees. Existing PoS agents have a 12-month window to align with the new registration process specified by licensees.

This measure aims to eliminate rogue PoS agents who engage in fraudulent practices, such as issuing SIM cards to antisocial or anti-national elements. The government has instructed that any existing PoS agents engaging in illegal activities will face termination and a three-year blacklist.”

It should therefore be one of the compliance requirements of every telecom operator to ensure that the POS agent displays the registration document that indicates that he is an authorized agent to issue SIM cards. 

Further, the mobile customers can check from time to time the number of SIM cards linked to them by verifying their number in https://tafcop.sancharsaathi.gov.in/telecomUser/

Currently upto 9 SIM cards are issued to a single person and bulk cards issue for companies are issued through an authorized signatory to be registered by the organizations with DOT.

PS: It is possible that most of the telecom companies have not introduced security measures as envisaged in the December 1, 2023 guideline and compliance auditors need to ensure that they specially check a sample of the retail stores to ensure that proper systems are in place at the SIM card issuing outlets.

Naavi

(continuation from the previous Article)

In order to look at the compliance requirements under EU-AI act, we need to first understand what is AI, what is the role of our organization under EU-AI act and what is the classification of the AI system.

We have discussed the definition of AI in our previous article.

In this article which may need some extensions, we shall explore the roles to which the EU-AI act is applicable, which is available through Article 1 .

Under Article 2.1 the Act is applicable to 

(a) providers placing on the market or putting into service AI systems or placing on the market general-purpose AI models in the Union, irrespective of whether those providers are established or who are located within the Union or in a third country;

(b) deployers of AI systems that have their place of establishment or who are located within the Union;

(c) providers and deployers of AI systems that have their place of establishment or who are located in a third country, where the output produced by the system is used in the Union;

(ca) importers and distributors of AI systems;

(cb) product manufacturers placing on the market or putting into service an AI system together with their product and under their own name or trademark;

(cc) authorised representatives of providers, which are not established in the Union;

(cc) affected persons that are located in the Union.

It is observed from the above scope that organizations who may be developers of AI in India whose products are accessible in EU either because the developer exports the products to importers in EU, or runs a website accessible from EU or directly operates in EU and offers the service may be coming under the Act.

Article 2.2 states that this Act is applicable in the following circumstances

For AI systems classified as high-risk AI systems in accordance with Articles 6(1) and 6(2) related to products covered by Union harmonisation legislation listed in Annex II, section B only Article 84 of this Regulation shall apply.

 Article 53 shall apply only insofar as the requirements for high-risk AI systems under this Regulation have been integrated under that Union harmonisation legislation.

However the subsection Article 2.3 states

“This Regulation shall not apply to areas outside the scope of EU law and in any event shall not affect the competences of the Member States concerning national security, regardless of the type of entity entrusted by the Member States to carry out the tasks in relation to those competences.”

“This Regulation shall not apply to AI systems if and insofar placed on the market, put into service, or used with or without modification of such systems exclusively for military, defence or national security purposes, regardless of the type of entity carrying out those activities.”

“This Regulation shall not apply to AI systems which are not placed on the market or put into service in the Union, where the output is used in the Union exclusively for military, defence or national security purposes, regardless of the type of entity carrying out those activities.”

The subsection Article 2.4 states

” This Regulation shall not apply to public authorities in a third country nor to international organisations falling within the scope of this Regulation pursuant to paragraph 1, where those authorities or organisations use AI systems in the framework of international cooperation or agreements for law enforcement and judicial cooperation with the Union or with one or more Member States, under the condition that this third country or international organisations provide adequate safeguards with respect to the protection of fundamental rights and freedoms of individuals.

the subsection Article 2.5 states

“This Regulation shall not affect the application of the provisions on the liability of intermediary service providers set out in Chapter II, Section 4 of Directive 2000/31/EC of the European Parliament and of the Council[*] [as to be replaced by the corresponding provisions of the Digital Services Act].”

[*] Directive 2000/31/EC of the European Parliament and of the Council of 8 June 2000 on certain legal aspects of information society services, in particular electronic commerce, in the Internal Market (‘Directive on electronic commerce’) (OJ L 178, 17.7.2000, p. 1)

The Article 2.5a states

This Regulation shall not apply to

AI systems and models, including their output, specifically developed and put into service for the sole purpose of scientific research and development.

(However) Union law on the protection of personal data, privacy and the confidentiality of communications applies to personal data processed in connection with the rights and obligations laid down in this Regulation.

This Regulation shall not affect Regulations (EU) 2016/679 and (EU) 2018/1725 and Directives 2002/58/EC and (EU) 2016/680, without prejudice to arrangements provided for in Article 10(5) and Article 54 of this Regulation.

The subsection Article 2. 5b states

5b. This Regulation shall not apply to any research, testing and development activity regarding AI systems or models prior to being placed on the market or put into service; those activities shall be conducted respecting applicable Union law. The testing in real world conditions shall not be covered by this exemption.

This Regulation is without prejudice to the rules laid down by other Union legal acts related to consumer protection and product safety.

The subsection Article 2 5c states

5c. This Regulation shall not apply to obligations of deployers who are natural persons using AI systems in the course of a purely personal non-professional activity.

The subsection Article 2 5e (5d missing) states

5e. This Regulation shall not preclude Member States or the Union from maintaining or introducing laws, regulations or administrative provisions which are more favourable to workers in terms of protecting their rights in respect of the use of AI systems by employers, or to encourage or allow the application of collective agreements which are more favourable to workers.

The subsection Article 2 5g (5f missing) states

5g. The obligations laid down in this Regulation shall not apply to AI systems released under free and open source licences unless they are placed on the market or put into service as high-risk AI systems or an AI system that falls under Title II and IV.

As one can observe,each of the above sub clauses need to be explored independently which we shall do so in subsequent articles.

If we look at how DGPSI tries to address similar concerns, we bank upon the legal provisions in India under ITA 2000 (Section 11) which provides that the actions of an automated system is attributable to the “Controller” of the system.

Hence under DGPSI if we identify any automated system which may be classified as AI or Generative AI etc., we try to identify whether the Data Fiduciary is in control of the means of processing. If he does not know what the AI code is doing, the AI developer or deployer is considered a “Joint Data Fiduciary” and all responsibilities for compliance of DPDPA 2023 lies with  him along with the liabilities for which the contract should provide an indemnity clause. Where the deployer is ready to disclose the code or give an assurance that the means of finance  is  exactly what the data fiduciary prescribes and is auditable by him, the deployer may be considered as a “Data Processor”.

DGPSI recommends business end-to end compliance and hence the issue of role definition for AI deployment from the perspective of the user data fiduciary should be settled at the time of approving a vendor or buying of a  software system.

Also, unlike EU-AI act which applies to both personal and non personal data, under DGPSI we look at systems which process personal data and in case of the developer or vendor we expect them to be able to certify that their AI is a “DPDPA Compliant AI”. The AI-DTS system is being developed as a system to evaluate the compliance maturity of the AI product.

(P.S: Readers would appreciate that these are pioneering concepts and are under development and continual improvement. The professional community is welcome to join hands with Naavi in developing these  concepts further.)

Naavi

(Continued from previous article: Impact of EU AI Act on India)

EU AI Act expects that “Providers”, “Deployers” and other “Trade intermediaries” shall be bound by the law (Effective from 2026).

The compliance to EU-AI act starts with flagging a “Product” or “Service” as “AI Product/Service”. Hence the definition of what constitutes an “AI System” becomes key to the compliance.

According to Article 3(1)

‘AI system‘ is a machine-based system designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments;

Though the compliance is based on the “Risk Assessment”, the applicability at the foundation stage is guided by this definition. (Other aspects of applicability are who is the user and where it is being used).

The ISO 420o1 is made applicable to an “AI management system” (AIMS) but does not clearly define how an organization can identify which part of its business is AI and which is not. It adopts ISO/IEC 22989:2022 for definition which vaguely defines Artificial intelligence as

“an engineered system set of methods or automated entities that together build, optimize and apply a physical, mathematical, or otherwise logical representation of a system, entity, phenomenon, process or data  so that the system can, for a given set of predefined tasks compute predictions, recommendations, or decisions”.

It is important for us to realize that at the umbrella level, all software systems are “Intelligent”. But the intelligence of a software system is limited to the instructions already embedded as the “Code”. The software therefore is a faithful servant that remembers the instructions and executes it based on the context.

An AI is a special software which has two key characteristics namely “Autonomy” and “Adaptiveness” as stated in the EU-AI definition. The ISO 42001 definition only speaks of “Predictions” and “Decisions” as terms that can be attributed to the nature of AI as different from other software.

In contrast the definitions arising from organizations like UNESCO which focus on the impact of AI on people recommend that

“AI systems are systems which have the capacity to process data and information in a way that resembles intelligent behaviour, and typically includes aspects of reasoning, learning, perception, prediction, planning or control.”

Taking these three definitions into consideration DGPSI adopts the following definition for AI.

Definition of AI under DGPSI

AI is a class of automated data processing system where the human intervention in decision output and application of decision to a business decision is below an acceptable threshold.

In order to define the threshold, three classes of AI are recognized as part of the definition.

Class 1:

Any software that has a Code correcting ability without the intervention of a human developer to generate an output is considered as an AI system-Class 1.

Class 2:

Any AI system that automatically implements a decision affecting a human is considered as AI system- Class 2

Class 3:

Any system that reacts to the human emotions, capable of creative outputs, including generative AI  and is considered as AI system- Class 3

This definition includes the “Autonomy” and “Adaptiveness” of the EU-AI act and “Predictiveness” , “Automated decision making” in the ISO 42001 and the “Human like behaviour” of the UNESCO definition. The sub classifications also take into account the “Risk Perceptions” such as “Unsupervised Machine Learning”, “Bias in automated decision making” and “Hallucination and rogue behaviour”.

The first task of compliance is therefore to label a software system as “AI System” using this definition.

PS: 

Under DGPSI (Digital Governance and Protection Standard of India) we define the DGPMS (Digital Governance and Protection Management System) and apply “Compliance By Design” principles. 

The DGPMS is an integration of PIMS, ISMS (Personal Data Processing) and AIMS (personal data processing). 

The AIMS under DGPMS is a sub system which is either a Joint Data Fiduciary or a Data Processor and compliance requirements are designed accordingly.

Comments are welcome.

Naavi