Data integrity — validation and verification
| English | Chinese | Pinyin |
|---|---|---|
| integrity | 完整性 | wán zhěng xìng |
| validation | 验证 | yàn zhèng |
| verification | 核对 | hé duì |
| range check | 范围检查 | fàn wéi jiǎn chá |
| check digit | 校验位 | jiào yàn wèi |
| double entry | 双重录入 | shuāng chóng lù rù |
| parity check | 奇偶校验 | jī ǒu jiào yàn |
| checksum | 校验和 | jiào yàn hé |
| cyclic redundancy check | 循环冗余校验 | xún huán rǒng yú jiào yàn |
| parity block check | 奇偶块校验 | jī ǒu kuài jiào yàn |
Every number passed every check
- In September 1999 NASA's Mars Climate Orbiter fired its engine to enter orbit and was never heard from again. It had flown 170 km lower than planned and burned up.
- One team's software produced thrust figures in pound-force seconds; the receiving software expected newton-seconds. Every value was a well-formed number in a sensible range. Every value was wrong by a factor of 4.45.
- No format check could have caught it, because the data was not malformed. It was simply not what it claimed to be. $327 million.
- This lesson is the two families of checks that guard integrity 完整性: validation, which asks whether data is sensible, and verification, which asks whether it was copied correctly. And the gap between them that Mars fell through.
Two questions
- Data has integrity when it is accurate, consistent and complete.
- Validation 验证 is an automatic check that entered data is reasonable and follows set rules. It asks: is this sensible?
- Verification 核对 checks that data has been entered or transferred correctly, by comparing it with the source or with a recomputed value. It asks: was this copied correctly?
- Neither proves the data is true. Both are needed.
Computing concept lab
Classify concrete examples by the computing idea they demonstrate.
Validation methods
| Check | Rule | Example |
|---|---|---|
| range check 范围检查 | between two limits | a month is 1 to 12 |
| limit check | on one side of a single limit | age ≥ 18 |
| existence (lookup) check | the value is in a list or table | the product code exists |
| length check | the right number of characters | a postcode of 6 characters |
| type / character check | the right kind of data | a phone field allows only digits |
| format check | matches a pattern | an email contains @ |
| presence check | required fields are not empty | a name is entered |
| check digit 校验位 | an extra digit computed from the others | ISBNs, card numbers |
Checking that a month entered is between 1 and 12 is a:
A range check confirms a value lies within sensible limits.
Match each validation check to its purpose.
Presence = not empty; format = matches a pattern; check digit = computed to catch mistyped digits.
An extra digit computed from the others to catch a mistyped number (as on ISBNs and card numbers) is a ______ digit.
A check digit is recomputed from the rest and compared, catching most single-digit and transposition errors.
Worked example: name the check the pseudocode performs
| Pseudocode | Check |
|---|---|
IF x < 0 OR x > 10 THEN OUTPUT "Invalid" |
range check: between two limits |
IF x = "" THEN OUTPUT "Invalid" |
presence check: not empty |
IF NOT(x = "Red" OR x = "Yellow" OR x = "Blue") THEN OUTPUT "Invalid" |
existence (lookup) check: one of a list |
IF LENGTH(x) <> 6 THEN OUTPUT "Invalid" |
length check: six characters |
IF MID(x, 1, 1) < "A" OR MID(x, 1, 1) > "Z" THEN OUTPUT "Invalid" |
format check: the first character is a letter |
- A car registration of one letter, three digits and two letters needs a format check on each position and a length check for six characters. A date of birth needs a format check (
DD/MM/YYYY), a range check (the month is 1 to 12, the year not in the future) and a presence check.
Match each line of pseudocode to the validation check it performs.
Empty test = presence; character count = length; two limits = range; a list of allowed values = existence or lookup.
Worked example: a check digit, and what it misses
- A simple scheme: the check digit is the remainder when the digit sum is divided by 10, appended to the number.
4162has digit sum 13, so it is stored as41623. - A user types
41523: the sum is now 12, the check digit should be 2, so the error is caught. - A user types
14623: the first two digits are swapped, but the sum is still 13, so the check digit matches and the error is not caught. Real schemes (ISBN, card numbers) weight each position so that a swap changes the result.
What validation cannot do
- Validation catches data that is wrongly formatted or out of range.
- It cannot catch data that has the right format and is factually wrong.
Bobtyped forBibpasses every check; so did every thrust figure sent to Mars. - For that you need verification against the source, and a person who reads what they typed.
Why can validation still let bad data through?
Validation checks the form, not the truth — a correctly formatted but wrong value passes.
Verification during data entry
- Double entry 双重录入: the data is entered twice, by the same person or by two people, and the computer compares the two versions and reports any difference. A new password is the everyday case.
- Visual check: the person compares what is on the screen with the original source document and corrects any difference before saving.
- Both protect integrity by making sure the stored data matches the source.
Asking a user to type a new password twice is an example of:
Double entry compares two typed copies to verify the data was entered correctly.
Verification during data transfer
- Bits can flip in transit. A parity check 奇偶校验 adds one extra bit to each byte so that the number of 1s is even (even parity) or odd; the receiver recounts, and a single flipped bit fails the count.
- A checksum 校验和 is a value calculated from a block of data by an algorithm and sent with it; the receiver recalculates it and compares. Transferring video files from a camera to a server works this way: a mismatch means retransmit.
- A cyclic redundancy check 循环冗余校验 (CRC) is a stronger checksum using polynomial division, catching many more error patterns.
One extra bit makes the count of 1s even; a flipped bit breaks it
Compute, send, recompute, compare
A parity check works by:
The parity bit makes the count of 1s even or odd; the receiver re-counts to detect a single-bit error.
Which are verification methods used during data transfer? Select all that apply.
Parity and checksums recompute a value at the receiving end. Double entry and a visual check happen during data entry.
A byte parity check detects every possible transmission error in that byte.
Two flipped bits leave the count of 1s unchanged, so they pass. Parity reliably catches single-bit errors only.
Worked example: a parity block locates the bit
- A parity block check 奇偶块校验 arranges the bytes in a grid: each byte carries a row parity bit, and one extra parity byte holds the column parity of the bytes above it.
- Four bytes are sent with even parity, followed by the parity byte. Count the 1s in every row and column: byte 3 has five and column 4 has three, both odd.
- The bit where that row and that column cross is the one that changed, so it is reset from 1 to 0. A single parity bit detects an error; a block locates and corrects it.
One row wrong, one column wrong: the bit is at the crossing
Put the steps of a parity block check in order.
Rows detect, columns detect, the crossing locates, and the flip corrects. That is what a block adds over a single parity bit.
Worked example: which check catches which error
- A user types their date of birth as
31/02/2009, then types their email address twice. Which check catches which error, and what is the difference? - Validation: a format or range check rejects
31/02/2009, because February never has 31 days. The computer tests the value against a rule. - Verification: double entry compares the two email addresses and reports a mismatch if they differ. The computer compares two copies, and it would accept a wrong address typed identically twice.
- Validation asks whether the data is sensible; verification asks whether it was entered correctly.
Sensible, or copied correctly: two different questions
Which statement is correct?
Validation tests sensibility against rules; verification tests that the data was entered/transferred without change.
Verification proves the data was copied or entered without change — but not that the value is factually correct.
Double entry / checksums prove the data arrived unchanged; only a human or a cross-check can confirm it is actually true.
Marks that slip away
- A check digit is validation on entry; parity and a checksum verify a transfer. Do not file them under the wrong heading.
- Neither check proves the data is true. Validation says "reasonable"; verification says "matches the source".
- A byte parity check misses two flipped bits in the same byte, because they cancel. Say "single-bit errors".
- Encryption is a security measure. It does nothing for integrity.
You've got it
- validation = automatic rules on entry: range, limit, existence, length, type, format, presence, check digit; it cannot catch right-format-but-wrong values
- verification on entry = double entry and a visual check; on transfer = parity (byte or block), checksum, CRC
- a parity block locates a flipped bit at the crossing of the failing row and column
- validation asks "sensible?"; verification asks "copied correctly?"; neither asks "true?"