Understanding Numeric Data in VBScript - Numeric Subtypes and Precision
(Page 2 of 5 )
Numeric data subtypes represent different levels of precision in these numbers and also control the precision of the numbers that are the result of VBScript’s calculations. Each data subtype allocates a specific amount of memory for a number. Thus, the precision of some numbers can become limited by the amount of memory available to them. Let’s take a look at the numeric subtypes.
Fixed width numeric subtypes |
Type | Precision | Data Range |
Byte | 1 byte | 0-255 |
Integer | 2 bytes | -32,768 to 32,767 |
Long | 4 bytes | -2,147,483,648 to 2,147,483,647 |
| | | |
Floating point numeric subtypes |
Single | 4 bytes | -3.402823E38 to -1.401298E-45 for negative values; 1.401298E-45 to 3.402823E38 for positive values |
Double | 8 bytes | -1.79769313486232E308 to -4.94065645841247E-324 for negative values; 4.94065645841247E-324 to 1.79769313486232E308 for positive values |
As you can see, the precision of each data subtype affects the respective data range dramatically. Let’s try to understand exactly what these precisions are and how they affect those ranges.
VBScript also provides two other special numeric types: Date and Currency. Date is a numeric representation of any valid date and Currency is used to force currency calculations.
Computer processors do not actually understand numbers. They work with bit patterns. Think of a bit as a switch that is either on or off, true or false. These bits are combined as strings, known as patterns, to indicate different values. These values are commonly represented as numbers with 0 being on, or true, and 1 being off, or false.
A bit represents a single, two value quantity like: on or off, true or false, 0 or 1.
Bit patterns, then, are combinations of bits. Here are some examples of patterns that can be constructed.
- A single bit has two states: either T or F. (21 possibilities)
- Two bits can distinguish between four states: TT, TF, FT, FF (22 possibilities)
- Three bits can distinguish between eight states: TTT, TTF, TFT, FTT, FFT, FTF, TFF, FFF. (23 possibilities)
As you can see, a bit pattern’s length is very important. The number of possible values for a bit pattern containing a given width is equal to 2n where n represents the number of bits in the pattern. This range is known as a pattern’s width.
Therefore, a bit pattern containing 10 bits would have a width of 210 or 1024 possibilities.
Computer storage is typically stated in bytes. A byte consists of eight bits. Using the formula above you can see that a byte has 28 or 256 possibilities. Thus, the Byte data type, with a precision of a single byte, has 256 possibilities. That’s why its range is 0-255. Remember that 0 is a state and that the Byte type only uses positive numbers.
Here again, the Integer type has a width of two bytes. (Remember that 2 bytes equals 16 total bits.) That means 216 or 65536 possibilities. Since the Integer type allows both positive and negative numbers, divide that by two—half in each direction. Thus you get 32,768 negative possibilities and 32,758 positive possibilities. The 0 state is considered a positive.
So, the Integer type has 32,768 negative possibilities, 0, and 32,757 positive possibilities greater than 0. In other words, its range is any integer between -32,768 and 32,757 including zero.
The floating point ranges are calculated in the same manner, however, there are bits used for the decimal point, so the ranges seem a bit limited compared to their fixed width counterparts.
Next: Number systems >>
More BrainDump Articles
More By Nilpo
