Download ANALYSIS OF ALGORITHMS

ANALYSIS OF ALGORITHMS
Data Structures
Algorithms
Definition
A step by step procedure for performing
some task in a finite amount of time
Analysis
Methodology for analyzing a “good”
algorithm (i.e. Archimedes)
Look at Running Time
How are algorithms described ?
PSEUDO-CODE
representation of an algorithm
combines
natural language
familiar programming language structures
facilitates the high-level analysis of an
algorithm
Pseudo-Code Conventions
Expressions
Algorithm Structures
Control Structures
Expressions
Standard math symbols
+
-
*
/
()
Relational operators
Boolean operators
and
or
not
Assignment operator
Array indexing A[i]
,=
Algorithm Structure
Algorithm heading
Algorithm name(param1, param2,...):
Input : input elements
Output : output elements
Statements
call
return statement
control structures
object.method(arguments)
return value
Control Structures
decision structures

if ... then ... [else ...]
while loops

while ... do
repeat loops

repeat ... until ...
for loop

for ... do
General Rules
communicate high level ideas and not
implementation details (programming
language specifics)
clear and informative
Example - Problem
Problem : Write pseudo-code for an
algorithm that will determine the
maximum element from a list (array)
Example - Algorithm
Algorithm arrayMax(A,n):
Input: An array A storing n integers.
Output: The maximum element in A.
currentMax
A[0]
for i
1 to n - 1 do
if currentMax < A[i] then
currentMax
A[i]
return currentMax
Analysis of Algorithms
Running time
depends on input size n
number of primitive operations performed
Primitive operation
unit of operation that can be identified in the
pseudo-code
Primitive Operations-Types
Assigning a value to a variable
Calling a method/procedure
Performing an arithmetic operation
Comparing two numbers
Returning from a method/procedure
Primitive Operations Some General Rules
for Loops
Nested for Loops
Conditional Statements
for Loops
The running time of a for loop is at most
the running time of the statements inside
the for loop (including tests) times the
number of iterations.
Example
for i
0 to n - 1 do
A[i]
0
Nested for Loops
Analyze these inside out. The total
running time of a statement inside a
group of nested for loops is the running
time of the statement multiplied by the
product of the sizes of all the for loops.
Example
for i
0 to n - 1 do
for j
0 to n - 1 do
k++
Conditional Statements
if (cond) then
S1
else
S2
The running time of an if-else statement is
never more than the running time of the
test plus the larger of the running times
of S1 and S2.
Primitive Operations Example
Count the number of primitive
operations in this program :
i=0
a=0
for i = 1 to n do
print i
a=a+i
return i
Primitive Operations Example
Estimate : 3n + 3
Linear Time
- (i.e. 6n + 2, 3n+5, n+1,etc.)
Primitive Operations Example 2
Count the number of primitive
operations in this program :
i=0
a=0
for i = 1 to n do
print i
for a = 1 to n do
print a
return i
Primitive Operations Example2
Estimate : n2 + n + 3
Polynomial (Quadratic) Time
Example 3 - Class
Algorithm sum(int n):
Input: Integer n
Output: Sum of the cubes of 1 to n
partial_sum
0
for i
1 to n do
partial_sum
partial_sum + i*i*i
return (partial_sum)
Primitive Operations Considerations
Arbitrariness of measurement
Worst-case versus average-case (difficult
to be exact : range)
Implicit operations
loop control actions
array access
Analysis of Algorithms
take-out the effect of constant factors
(i.e. 2n vs 3n : same level)
this can be attributed to differences in
hardware and software environments
compare algorithms across types (linear
vs polynomial vs exponential)
look at growth rates
Asymptotic Notation
Goal:
To simplify analysis by getting rid of
irrelevant information
The “Big-Oh” Notation
Given functions f(n) and g(n), f(n) is O
(g(n)) if f(n) <= c g(n) for n>= n0 where c
and n0 are constants
Big-Oh Notation
Rule : Drop lower order terms and
constant factors
5n + 2 is O (n)
4n3log n + 6n3 + 1 is O (n3logn)
Big-Oh Notation - Relatives
Big Omega (reverse)
Big Theta (both ways, same level)
General Rule : Use the most descriptive
notation to describe the algorithm
Practical Significance of
Big-Oh Notation
Example
for i
0 to n-1 do
A[i]
0
Running time is
2n, if we count assignments & array accesses
3n, if we include implicit loop assignments
4n, if we include implicit loop comparisons
Regardless, running time is O (n)
Algorithms - Common
Classifications
Typical functions that classify algorithms
constant
logarithmic
linear
quadratic
exponential
O (1)
O (log n)
O (n)
O (n2)
O (an), n > 1
Linear Search vs Binary
Search
Linear Search
Binary Search
O (n)
O (log n)
Example : to search for a number from
1024 entries using the binary search
algorithm will only take 10 steps
O(log n) Algorithm
Steps
Range (n)
Power of 2
0
1
20
1
2
21
2
4
22
3
8
23
4
16
24
5
32
25
6
64
26
7
128
27
O(log n) Algorithm
Binary Search
Algorithm bin_search(int a[], int x, int n)
Input: Array a, search target x,
array size n
Output: Position of x in the array (if found)
`
{
low
0; high
n-1
while (low <= high) do
mid
(low + high)/2
if(a[mid] < x) then
low
mid + 1
else
if (a[mid] > x) then
high
mid - 1
else
return (mid)
return (NOT_FOUND)
}
Prefix Average Problem
Given an array X storing n numbers, we
want to compute an array A such that
A[1] is the average of elements X[0] …
X[I] for I = 0 … n-1.
algorithm A = O (n2)
algorithm B = O (n)