Sparse and Redundant Representations

A long long time ago, echoing philosophical and aesthetic principles that existed since antiquity, William of Ockham enounced the principle of parsimony, better known today as Ockham's razor: "Entities should not be multiplied without neces sity. " This principle enabled scientists to select the "best" physical laws and theories to explain the workings of the Universe and continued to guide scienti?c research, leadingtobeautifulresultsliketheminimaldescriptionlength approachtostatistical inference and the related Kolmogorov complexity approach to pattern recognition. However, notions of complexity and description length are subjective concepts anddependonthelanguage"spoken"whenpresentingideasandresults. The?eldof sparse representations, that recently underwent a Big Bang like expansion, explic itly deals with the Yin Yang interplay between the parsimony of descriptions and the "language" or "dictionary" used in them, and it became an extremely exciting area of investigation. It already yielded a rich crop of mathematically pleasing, deep and beautiful results that quickly translated into a wealth of practical engineering applications. You are holding in your hands the ?rst guide book to Sparseland, and I am sure you'll ?nd in it both familiar and new landscapes to see and admire, as well as ex cellent pointers that will help you ?nd further valuable treasures. Enjoy the journey to Sparseland! Haifa, Israel, December 2009 Alfred M. Bruckstein vii Preface This book was originally written to serve as the material for an advanced one semester (fourteen 2 hour lectures) graduate course for engineering students at the Technion, Israel.

Introduces theoretical and numerical foundations before tackling applications

Preface.- Part I. Theoretical and Numerical Foundations.- 1. Introduction.- 2. Uniqueness and Uncertainty.- 3. Pursuit Algorithms - Practice.- 4. Pursuit Algorithms - Guarantees.- 5. From Exact to Approximate Solution.- 6. Iterated Shrinkage Algorithms.- 7.Towards Average Performance Analysis.- 8. The Danzig Selector Algorithm.- Part II. Signal and Image Processing Applications.- 9. Sparsity-Seeking Methods in Signal Processing.- 10. Image Deblurring - A Case Study.- 11. MAP versus MMSE Estimation.- 12. The Quest For a Dictionary.- 13. Image Compression - Facial Images.- 14. Image Denoising.- 15. Other Applications.- 16. Concluding Remarks.- Bibliography.- Index

This book introduces sparse and redundant representations with a focus on applications in signal and image processing. It details mathematical modeling for signal sources along with how to use the model for tasks such as denoising, restoration and separation.