A TOR (terms of reference, the document that tells bidders what the organization needs) and an RFP (request for proposal, an invitation for vendors to propose a solution and a price) decide what security system the organization ends up with. Written too loosely, they produce something that passes on paper and fails in use. Written too narrowly, they leave only one vendor. This article covers how to write a measurable TOR and how to plan a PoC (proof of concept, a time-limited trial of the real system against criteria agreed in advance). It is not legal advice.
Write the TOR around outcomes, not brands
The rule that works for any project is to say what the system must do and how it will be measured, not whose product it must be. For Thai government bodies, the Public Procurement and Supplies Administration Act B.E. 2560 (2017) sets the principles of value for money, transparency and fair competition. We could not check the original wording on naming brands or the details of the related Ministry of Finance regulation, so this article does not summarize them. Confirm with your procurement office and legal counsel before relying on it. Our view is that a security TOR should contain the following items.
- Problem and goal: say in a few lines what problem is being solved, for example faster detection, and which number will show success
- Size of the work: state the number of users, machines and daily log volume to support, with the unit of measure made clear
- Mandatory versus preferred: separate what cannot be missing from what is an advantage, and do not mix them
- Integration with existing systems: state what it must connect to, such as the identity system, firewalls or an existing SIEM (a system that collects and analyzes logs from many sources)
- After-sales service: define installation, training and the SLA (service level agreement, such as response and fix times when problems occur)
- Acceptance criteria: write down at the start what the system must do before the work is accepted
Requirements that often lead to the wrong product
Most problems come from wording that can be read several ways. Before writing, know which type of tool is needed. If the differences between these systems are unclear, read SIEM + SOAR vs XDR: strengths, limits and recommendations first. This list is also our view.
- Wording too broad to measure: a line such as "must use AI for detection" cannot be accepted or rejected; say which behavior must be detected and how it will be proven
- Copying one vendor's specification: requirements that match one brand's datasheet word for word keep other vendors out and risk complaints
- Counting in different units: some vendors price by events per second (EPS), others by data volume per day; if the TOR names no unit, prices cannot be compared
- No integration requirements: a system that works well alone may not connect to what already exists, leading to extra cost later
- No acceptance criteria: with no agreement on what counts as "working", delivery turns into an argument
Open to interpretation
- Must use AIcannot be accepted or rejected
- Supports lots of logsno number or unit
- Works with existing systemsdoes not say which systems
Measurable and provable
- Detects named behaviorsa list and a way to prove it
- Log volume in a stated unitprices can be compared
- Connects to named systemstestable in the PoC
A PoC that gives answers sets its criteria first
A PoC runs the system in conditions close to the real ones to see whether it does what the vendor claims. The sources we found on this are vendor and tech-press articles, not formal standards. They agree on a few points: pass and fail criteria must be agreed in writing before testing starts, the time must be limited, and testing should use a setup close to production rather than an isolated lab with nothing connected. A Forrester analyst quoted by TechTarget suggests using 3-6 test scenarios. Our view is that a PoC plan should contain the following items.
- Criteria as numbers: for example, how many of the test scenarios are detected and the maximum days to set up
- Real data and systems: use logs and connections close to real operating conditions
- 3-6 test scenarios: include both common and difficult cases
- Real users take part: the people who will use the system every day do the scoring
- Results recorded as pass, partial or fail: note every item with the problems found, not only the ones that passed
To test detection without affecting live systems, see the types and safety rules for simulation tools in Network & Cyber Security Simulator: types and use. Run it in an environment separated from production.
A good PoC is not one vendor's demonstration. If a PoC is part of a procurement, state the conditions in the documents, give every bidder the same scenarios, criteria and time, and decide from the recorded results. Whether a PoC may be made a condition of a Thai government bid, and by what method, we have not checked against the relevant regulations. Ask your procurement office and legal counsel.
Where to start
- This week: write a one-page statement of the problem and the number that defines success, before talking to any vendor
- Before drafting the TOR: count the real workload and fix the unit of measure
- While drafting: separate mandatory from preferred items and check that no requirement matches only one brand
- Before the PoC starts: have stakeholders sign the pass and fail criteria in writing
If you are not sure which gaps the organization needs to buy for, try the self-assessment for an overview before writing the TOR.