Can you use ChatGPT for security questionnaires?

Yes, you can use ChatGPT to draft answers to a security questionnaire. But a questionnaire isn't a writing task. Each answer is a factual claim about your company, and the buyer may write it into a contract. The risks come from that gap between a fluent answer and a true one.

The real risks

1. Answers that sound right but aren't true

Ask a chatbot "Do you perform quarterly access reviews?" and it may write a confident Yes with a nice paragraph. It doesn't know whether you do. Language models are good at producing what a typical answer looks like. That's exactly the wrong thing here, because the typical company in the training data may be bigger and more mature than yours.

Security practitioners warn about the same thing with any tool. In a r/cybersecurity thread (archived copy) on answering questionnaires, commenters listed boilerplate that overstates maturity and answering from intent instead of evidence as common mistakes. Commenters also said that facts like encryption and SOC 2 status shouldn't be auto-generated.

2. Citations you can't verify

You can ask ChatGPT to cite your policy. It may give you a quote. Unless you check, you don't know if that quote appears in your document word for word, or if it's a paraphrase or an invention dressed as a quote. Nothing in a normal chat checks this for you.

3. Where your policies go

Your security policies are confidential. They describe your defenses. What happens to them depends on which OpenAI product you use:

• Personal ChatGPT accounts: OpenAI says that for services for individuals, it may use your content to train its models. You can opt out in Data Controls, which applies to new conversations. Feedback is an exception. If you rate a response, OpenAI says the whole conversation may be used for training.
• Temporary Chat: OpenAI says a temporary chat is not used to improve its models while it stays temporary. It may keep a copy for up to 30 days for safety.
• ChatGPT Business, Enterprise and the API: OpenAI says that by default it does not train on inputs or outputs from business products, unless the organization opts in. Its enterprise privacy page covers retention, ownership and admin controls.

Also check your own commitments. If you signed an NDA with the buyer, their questionnaire may be confidential too.

4. The Excel file

Questionnaires arrive as Excel files with dropdowns, merged cells, hidden columns and validation. If you ask a chatbot to return a filled spreadsheet, check that it's the same file. Look for missing dropdowns, changed questions, shifted rows and lost formatting. Many buyers' review tools expect their exact template. The safer path is to paste answers into the original file yourself.

How to use ChatGPT more safely

1. Use the right plan. Use a business plan, or turn off training in Data Controls. Don't upload anything to an account you don't control.
2. Give it only your documents. Paste your policies and tell it to answer only from them. Don't let it use general knowledge.
3. Require a quote for every answer. Ask for the exact sentence from your document and the document name.
4. Allow "No proof found". Tell it explicitly that if the documents don't answer the question, it must say so instead of guessing.
5. Check every quote. Search your document for each quoted sentence (Ctrl+F). If it isn't there word for word, the answer is unproven.
6. Review the sensitive rows yourself. Breaches, subprocessors, data location, legal commitments.
7. Fill the original file. Copy answers into the buyer's spreadsheet. Don't send a regenerated one.

A prompt along these lines helps:

Answer each question using only the policy text below. For every answer, give the exact sentence you relied on, copied word for word, and the document name. If the text does not answer the question, reply "No proof found". Do not use outside knowledge. Do not assume a control exists.

Even with this prompt, step 5 is the one that matters. A prompt asks for good behavior. It doesn't check it.

What we'd test

We haven't run this test yet, so we're not reporting results. Here is how we'd measure whether a chatbot is safe for this job. When we run it, we plan to publish the data and scripts so anyone can repeat it.

Data. A public company's security policies and its own completed questionnaire as the answer key. For example, a CAIQ v4 (261 questions) with the company's published answers. We'd add questions the policies clearly don't answer, so the right response is "No proof found".

Setup. The same documents and the same prompt (above) for every tool. Fresh sessions. Temperature and model version recorded. Each run done three times, because answers vary between runs.

What we'd measure:

• Accuracy: of the answers given, how many match the company's own answer.
• Invented quotes: how many "quotes" don't appear word for word in the documents. Checked by code, not by eye.
• Unsupported Yes: on questions the documents don't answer, how often the tool answers Yes anyway.
• Abstention: how often it correctly says "No proof found".
• File integrity: whether the returned Excel file keeps its questions, layout and dropdowns.

What would count as passing. Zero invented quotes and zero unsupported Yes answers. Accuracy matters less than those two. A blank answer costs you ten minutes. A false Yes can cost you the deal or a contract dispute.

How Citesheet helps

Citesheet fills Excel questionnaires, such as the CAIQ, from your own policies, and code checks that every quote exists word for word in your document before the answer ships; otherwise the row says "No proof found". On a 261-question CAIQ, 158 of 163 filled answers matched GitLab's own answers, with 0 invented quotes (benchmark).