机器学习题库.docxVIP

机器学习题库 一、极大似然 1、 ML estimation of exponential model (10) A Gaussian distribution is often used to model data on the real line, but is sometimes inappropriate when the data are often close to zero but constrained to be nonnegative. In such cases one can fit an exponential distribution, whose probability density function is given by bp ?x?? 1 e? x b b Given N observations xidrawn from such a distribution: Write down the likelihood as a function of the scale parameter b. Write down the derivative of the log likelihood. Give a simple expression for the ML estimate for b. 2、换成 Poisson 分布： p ?x | ? ?? ? xe??, y ? 0,1,2,... x! l ?? ?? ?N log ?p ?x i | ? ??? ?N x log? ?? ? log ?x!? i i?1 i?1 ?? ???N ? i?1 x ?? log? ? N? ? ?N i ?i? i ? log ?x !? i 二、贝叶斯 1、 贝叶斯公式应用 假设在考试的多项选择中，考生知道正确答案的概率为 p，猜测答案的概率为 1-p，并且假设考生知道正确答案答对题的概率为 1，猜中正确答案的概率为1m ，其中 m 为多选 项 的 数 目 。 那 么 已 知 考 生 答 对 题 目 ， 求 他 知 道 正 确 答 案 的 概 率 。： p ?known | correct ?? p ?known, correct ? ? p p ?known ??1 p ? ?1? p ? m 2、 Conjugate priors Given a likelihood p ?x | ? ? for a class models with parameters θ, a conjugate prior is a distribution p ?? | ? ? with hyperparameters γ, such that the posterior distribution p ?? | X ,? ?? ? p ?X | ? ? p ?? | ? ?? p ?? | ? ?? 与先验的分布族相同 Suppose that the likelihood is given by the exponential distribution with rate parameter λ: p ?x | ??? ?e??x Show that the gamma distribution Gamma ?? | ?, ? ?? ?? ? ?? ? ???1e? ?? _ is a conjugate prior for the exponential. Derive the parameter update given observations x , , x 1 N and the prediction distribution p ?x  N ?1 | x , , x , x ?. Show that the beta distribution is a conjugate prior for the geometric distribution p ?x ? k | ? ?? ?1?? ?k ?1 ? which describes the number of time a coin is tossed until the first heads appears, when the probability of heads on each toss is θ. Derive the parameter update rule and prediction distribution. Suppose p ?? | ? ? is a conjugate prior for the likelihood p?x | ? ?; show that the mixture prior