Download crt

Applied Symbolic Computation
(CS 680/480)
Lecture 6: Integer Multiplication, Interpolation, and the
Chinese Remainder Theorem
Jeremy R. Johnson
May 9, 2001
May 9, 2001
Applied Symbolic Computation
1
Introduction
• Objective: To show the relationship between the Chinese
Remainder Theorem and Interpolation. To derive a series of
fast integer multiplication algorithms using interpolation.
–
–
–
–
–
–
–
Karatsuba’s Algorithm
Polynomial algebra
Polynomial version of the Chinese Remainder Theorem
Interpolation
Vandermonde Matrices
Toom-Cook algorithm
Polynomial multiplication using interpolation
References: Lipson, Cormen et al.
May 9, 2001
Applied Symbolic Computation
2
Karatsuba’s Algorithm
• Using the classical pen and paper algorithm two n digit
integers can be multiplied in O(n2) operations. Karatsuba
came up with a faster algorithm.
• Let A and B be two integers with
– A = A110k + A0, A0 < 10k
– B = B110k + B0, B0 < 10k
– C = A*B = (A110k + A0)(B110k + B0)
= A1B1102k + (A1B0 + A0 B1)10k + A0B0
Instead this can be computed with 3 multiplications
• T0 = A0B0
• T1 = (A1 + A0)(B1 + B0)
• T2 = A1B1
• C = T2102k + (T1 - T0 - T2)10k + T0
May 9, 2001
Applied Symbolic Computation
3
Complexity of Karatsuba’s Algorithm
• Let T(n) be the time to compute the product of two n-digit
numbers using Karatsuba’s algorithm. Assume n = 2k.
T(n) = (nlg(3)), lg(3)  1.58
• T(n)  3T(n/2) + cn
 3(3T(n/4) + c(n/2)) + cn = 32T(n/22) + cn(3/2 + 1)
 32(3T(n/23) + c(n/4)) + cn(3/2 + 1)
= 33T(n/23) + cn(32/22 + 3/2 + 1)
…
 3iT(n/2i) + cn(3i-1/2i-1 + … + 3/2 + 1)
...
 cn[((3/2)k - 1)/(3/2 -1)] --- Assuming T(1)  c
 2c(3k - 2k)  2c3lg(n) = 2cnlg(3)
May 9, 2001
Applied Symbolic Computation
4
Divide & Conquer Recurrence
Assume T(n) = aT(n/b) + (n)
• T(n) = (n)
[a < b]
• T(n) = (nlog(n)) [a = b]
• T(n) = (nlogb(a))
May 9, 2001
[a > b]
Applied Symbolic Computation
5
Polynomial Algebra
• Let F[x] denote the set of polynomials in the variable x
whose coefficients are in the field F.
• F[x] becomes an algebra where +, * are defined by
polynomial addition and multiplication.
m
n
A( x)   ai x , B( x)   b j x
i
i 0
j
j 0
m n
C ( x)  A( x) B( x)   ck x ,
k
k 0
c
May 9, 2001
k

a b
k i  j
i
j

min( k , m )
a b
i
i  max( 0 , k  n )
Applied Symbolic Computation
k i
6
Polynomial Algebra mod a Polynomial
• A(x)  B(x) (mod f(x))  f(x)|(A(x) - B(x))
• This equivalence relation partitions polynomials in F[x] into
equivalence classes where the class [A(x)] consists of the
set {A(x) + k(x)f(x), where k(x) and f(x) are in F[x]}.
• Choose a representative for [A(x)] with degree < deg(f(x)).
Can choose rem(A(x),f(x)).
• Arithmetic
– [A(x)] + [B(x)] = [A(x) + B(x)]
– [A(x)] * [B(x)] = [A(x)B(x)]
• The set of equivalence classes with arithmetic defined like
this is denoted by F[x]/(f(x))
May 9, 2001
Applied Symbolic Computation
7
Modular Inverses
Definition: B(x) is the inverse of A(x) mod f(x), if A(x)B(x)  1
(mod f(x))
The equation A(x)B(x)  1 (mod f(x)) has a solution iff
gcd(A(x),f(x)) = 1.
In particular, if f(x) is irreducible, then F[x]/(f(x)) is a field.
By the Extended Euclidean Algorithm, there exist u(x) and v(x)
such that A(x)u(x) + B(x)v(x) = gcd(A(x),f(x)). When
gcd(A(x),f(x)) = 1, we get A(x)v(x) + f(x)v(x) = 1. Taking this
equation mod f(x), we see that A(x)v(x)  1 (mod f(x))
May 9, 2001
Applied Symbolic Computation
8
Polynomial Version of the Chinese
Remainder Theorem
Theorem: Let f(x) and g(x) be polynomials in F[x] (coefficients
in a field). Assume that gcd(f(x),g(x)) = 1. For any A1(x) and
A2(x) there exist a polynomial A(x) with A(x)  A1(x) (mod
f(x)) and A(x)  A2(x) (mod g(x)).
Theorem: F[x]/(f(x)g(x))  F[x]/(f(x))  F[x]/(g(x)). I.E. There is a
1-1 mapping from F[x]/(f(x)g(x)) onto F[x]/(f(x))  F[x]/(g(x))
that preserves arithmetic.
A(x)  (A(x) mod f(x), A(x) mod g(x))
May 9, 2001
Applied Symbolic Computation
9
Constructive Chinese Remainder
Theorem
Theorem: If gcd(f(x),g(x)) = 1, then there exist Ef(x) and Eg(x)
(orthogonal idempotents)
–
–
–
–
Ef(x) 
Ef(x) 
Eg(x) 
Eg(x) 
1 (mod f(x))
0 (mod g(x))
0 (mod f(x))
1 (mod g(x))
It follows that A1(x) Ef(x) + A2(x) Eg(x)  A1(x) (mod f(x)) and 
A2(x) (mod g(x)).
Proof.
Since gcd(f(x),g(x)) = 1, by the Extended Euclidean Algorithm,
there exist u(x) and v(x) with f(x)u(x) + g(x)v(x) = 1. Set
Ef(x) = g(x)v(x) and Eg(x) = f(x)u(x)
May 9, 2001
Applied Symbolic Computation
10
Interpolation
• A polynomial of degree n is uniquely determined by its
value at (n+1) distinct points.
Theorem: Let A(x) and B(x) be polynomials of degree m. If
A(i) = B(i) for i = 0,…,m, then A(x) = B(x).
Proof.
Recall that a polynomial of degree m has m roots.
A(x) = Q(x)(x- ) + A(), if A() = 0, A(x) = Q(x)(x- ), and deg(Q)
= m-1
Consider the polynomial C(x) = A(x) - B(x). Since C(i) = A(i) B(i) = 0, for m+1 points, C(x) = 0, and A(x) must equal B(x).
May 9, 2001
Applied Symbolic Computation
11
Lagrange Interpolation Formula
• Find a polynomial of degree m given its value at (m+1)
distinct points. Assume A(i) = yi

( x  j ) 
y
A( x)  i 0  
 j i (  )  i
i
j 

m
• Observe that
 ( k  j ) 
y 
A( k )  i 0  
 j i (  )  i
i
j 

m
May 9, 2001
Applied Symbolic Computation
y
k
12
Polynomial Evaluation, Interpolation
and the CRT
• Since A(x) = Q(x)(x- ) + A(), A(x)  A() (mod (x- ))
• If   , then gcd((x- ),(x- )) = 1. Therefore, we can apply
the CRT to find a quadratic polynomial A(x) such that A() =
y and A() = z.
• A(x) = A() E(x) + A() E(x), where
– E(x) = (x - )/( - )
– E(x) = (x - )/( - )
• Observe that
– E() = 1 and E() = 0, so that E(x)  1 (mod (x- )) and E(x)  0 (mod
(x- )). The equivalent results hold for E(x)
May 9, 2001
Applied Symbolic Computation
13
Multifactor CRT
• The CRT can be generalized to the case when we have n
pairwise relatively prime polynomials. If f1(x),…,fn(x) are
pairwise relatively prime, i.e. gcd(fi(x),fj(x)) = 1 for i  j, then
given A1(x),…,An(x) there exists a polynomial A(x) such that
A  Ai(x) (mod fi(x)).
• Moreover, there exist a system of orthogonal idempotents:
E1(x),…,En(x), such that Ei(x)  1 (mod fi(x)) and Ei(x)  0
(mod fj(x)) for i  j.
• A(x) = A1(x)E1(x) + … + An(x)En(x)
May 9, 2001
Applied Symbolic Computation
14
Lagrange Interpolation and the CRT
• Assume that 0, 1,…, n are distinct and let fi(x) = (x- i).
Then gcd(fi(x),fj(x)) = 1 for i  j.
• Let
( x  j )
E ( x)   (
i
j i
  )
i
j
• Then Ei(x)  1 (mod fi(x)) and Ei(x)  0 (mod fj(x)) for i  j,
and Lagrange’s interpolation formula is
• A(x) = A (0)E0(x) + … + A(n )En(x)
May 9, 2001
Applied Symbolic Computation
15
Matrix Version of Polynomial
Evaluation
• Let A(x) = a3x3 + a2x2 + a1x + a0
• Evaluation at the points , , ,  is obtained from the
following matrix-vector product

 A( )  1
 A(  ) 1


 A( )  1

 
 A( )  1

May 9, 2001




1
1
1
1




2
2
2
2
 
 3 a0
 

  a1
3
a 

  2
3
a3 

Applied Symbolic Computation
3
16
Vandermonde Matrix
1

1
V ( 0 ,..., n)  
...

 1


1
0
1
1
...

 0n
...  1 
... ... 

n
...  n 
...
n

det(V )   (  )
i j
1
n
j
i
V(0,…, n) is non-singular when 0,…, n are distinct.
May 9, 2001
Applied Symbolic Computation
17
Matrix Interpretation of Interpolation
• Let A(x) = anxn + … + a1x +a0 be a polynomial of degree n.
The problem of determining the (n+1) coefficients an,…,a1,a0
from the (n+1) values A(0),…,A(n) is equivalent to solving
the linear system
 A( 0)   1

 
 A( 1)    1
 ...  ...

 
 A( n)  1
May 9, 2001


1
0
1
1
...

1
n
...
...
...
  a 
   a 
n
0
n
0
1
1
...  ... 


n 
...  n  a3
Applied Symbolic Computation
18
Matrix Interpretation of Interpolation
• The previous system has a solution when the 0,…, n are
distinct. The solution can be obtained using Lagrange
interpolation. In fact, the inverse of V(0,…, n) is obtained
from the idempotents
( x  j )
E ( x)   (
i
May 9, 2001
j i
  )
i
j
Applied Symbolic Computation
19
Polynomial Multiplication using
Interpolation
• Compute C(x) = A(x)B(x), where degree(A(x)) = m, and
degree(B(x)) = n. Degree(C(x)) = m+n, and C(x) is uniquely
determined by its value at m+n+1 distinct points.
• [Evaluation] Compute A(i) and B(i) for distinct i,
i=0,…,m+n.
• [Pointwise Product] Compute C(i) = A(i)*B(i) for
i=0,…,m+n.
• [Interpolation] Compute the coefficients of C(x) = cnxm+n +
… + c1x +c0 from the points C(i) = A(i)*B(i) for i=0,…,m+n.
May 9, 2001
Applied Symbolic Computation
20
Interpolation and Karatsuba’s
Algorithm
• Let A(x) = A1x + A0, B(x) = B1x + B, C(x) = A(x)B(x) = C2x2 +
C 1x + C 0
• Then A(10k) = A, B(10k) = B, and C = C(10k) = A(10k)B(10k) =
AB
• Use interpolation based algorithm:
–
–
–
–
Evaluate A(), A(), A() and B(), B(), B() for  = 0,  = 1, and  =.
Compute C() = A()B(), C() = A() B(), C() = A()B()
Interpolate the coefficients C2, C1, and C0
Compute C = C2102k + C110k + C0
May 9, 2001
Applied Symbolic Computation
21
Matrix Equation for Karatsuba’s
Algorithm
• Modified Vandermonde Matrix
• Interpolation
 C (0)  1 0 0 c0 
 C (1)   1 1 1  

 
  c1 
C () 0 0 1 c2 
c0   1 0 0  

A
0 B0
  



 c1    1 1  1  A0  A1B0  B1
c   0 0 1  

A
B
1
1


 2
May 9, 2001
Applied Symbolic Computation
22
Integer Multiplication Splitting the
Inputs into 3 Parts
• Instead of breaking up the inputs into 2 equal parts as is
done for Karatsuba’s algorithm, we can split the inputs into
three equal parts.
• This algorithm is based on an interpolation based
polynomial product of two quadratic polynomials.
• Let A(x) = A2x2 + A1x + A0, B(x) = B2x2 + B1x + B, C(x) =
A(x)B(x) = C4x4 + C3x3 + C2x2 + C1x + C0
• Thus there are 5 products. The divide and conquer part still
takes time = O(n). Therefore the total computing time T(n) =
5T(n/3) + O(n) = (nlog3(5)), log3(5)  1.46
May 9, 2001
Applied Symbolic Computation
23
Asymptotically Fast Integer
Multiplication
• We can obtain a sequence of asymptotically faster
multiplication algorithms by splitting the inputs into more
and more pieces.
• If we split A and B into k equal parts, then the
corresponding multiplication algorithm is obtained from an
interpolation based polynomial multiplication algorithm of
two degree (k-1) polynomials.
• Since the product polynomial is of degree 2(k-1), we need to
evaluate at 2k-1 points. Thus there are (2k-1) products. The
divide and conquer part still takes time = O(n). Therefore
the total computing time T(n) = (2k-1)T(n/k) + O(n) =
(nlogk(2k-1)).
May 9, 2001
Applied Symbolic Computation
24
Asymptotically Fast Integer
Multiplication
• Using the previous construction we can find an algorithm
to multiply two n digit integers in time (n1+ ) for any
positive .
– logk(2k-1) = logk(k(2-1/k)) = 1 + logk(2-1/k)
– logk(2-1/k)  logk(2) = ln(2)/ln(k)  0.
• Can we do better?
• The answer is yes. There is a faster algorithm, with
computing time (nlog(n)loglog(n)), based on the fast
Fourier transform (FFT). This algorithm is also based on
interpolation and the polynomial version of the CRT.
May 9, 2001
Applied Symbolic Computation
25