Test Plan
All empaneled certifiers are expected to write test toolkits that adhere strictly to the Sahamati Test Plan. The toolkit should not add or remove test scenarios, on its own. Any suggestions for improvement to the test plan are welcome and should be sent to Sahamati directly.
This Test Plan is a collection of test scenarios, grouped by FIP, FIU, AA participant roles.
The purpose of the Test Plan is to provide a common, technical basis for assessing and guaranteeing interoperability of the certified entity with other participants in the ecosystem.
The empaneled certifiers must contribute their time to ensuring the Test Plan is kept current and aligned with any changes published by the standards authority, from time to time. Their toolkits consequently also must be kept up-to-date with the Test Plan.
Validity of Assessment
The assessment of conformance is done for a particular version of the ReBIT Technical Specification and as per the scope defined in a particular version of the Sahamati Certification Framework.
The certificate awarded, post the assessment, is valid till the time one of the following events happen:
- The Technical Specification version is sunset by the standards authority (i.e. ReBIT), following the release of a new version
- The Sahamati Certification Framework version is sunset by Sahamati following the release of a new version of the Certification Framework.
- The certified entity implements a newer version of the Technical Specification, in production, in advance of the previous version being sunset
- The certified entity makes any change to the code – bug-fixes, feature changes, in production.
Any change to an existing version, whether major, minor or a bug-fix, is deemed to be a “new” version – for now.
When the Test Plan is upgraded to be able to check backward and forward-compatibility of implementations of the standard, the validity of conformance assessment will remain unaffected by minor- or bug-fix changes to a version.
Certified entities will then have to go through a re-assessment only when a major version change is announced by the standards authority.
The certificate awarded thus, will NOT have a validity expiry date. It only indicates the VERSION of the technical standard that the entity conforms to.
Frequency of Re-Certification
Once assessed, a certified entity needs to undergo a re-assessment only when one of the four events, as mentioned above, is detected.
Events #1 and #2, i.e. the sunset of the ReBIT Technical Specifications or the sunset of the Sahamati Certification Framework, are detected based on dates published by ReBIT / Sahamati in advance.
Events #3 and #4, i.e. changes made in production by entities, are detected based on a periodic self-test done by each certified entity, as described below. If the self-test fails and the reason for the same is traced to one of the two types of events (#3 and #4), a recertification run is required to be performed.
Frequency of Self Test
Periodicity: Quarterly
Process: Declaration of production environment compliance by TSP/FIU/FIP/AAs + re-run against the empaneled certifier’s testing tool.
A time-triggered frequency of continued compliance (not needing re-certification but just a self-test-driven submission of reports and a declaration) is required, as part of the Sahamati Certification Framework.
This involves each certified entity running the empaneled certifier’s tool, on a quarterly basis, in the environment dedicated to running the certification. (As of now, this is a UAT environment).
In addition, the certified entity must declare, on a quarterly basis, that the production environment has the same code as the environment in which certification has been done.
The benefit of this process is to help participants detect if any changes made to production systems have resulted in breakage in the code that grants compliance to technical standards. The Test Plan’s scenarios, if successfully passed, provides a guarantee that there are no non-compliances introduced on account of new changes in production code.
Liability
The Sahamati Certification Framework acts as a strong supplement to the ReBIT Technical Standards and provides assurance of interoperable API communication. This framework is built and maintained with the active participation of the community of Technology Service Providers and empaneled certifiers.
It thus confers the following benefits:
- Avoidance of downstream costs and efforts of issue-resolution for developers, if they occur in a “live” environment
- Avoidance of reputation loss, owing to issues resulting in denial of service to customers
- Mitigation of compliance risks arising due to customer grievances or disputes from other ecosystem participants
However, neither Sahamati nor the empaneled certifiers or the intermediate certified entities, bear any liability for interoperability issues that may still arise, post the conformance assessment.