本文共 3444 字,大约阅读时间需要 11 分钟。
目的:用户的购买行为看起来是没有规律可循的,但其实从时间有序的角度看,也许是有规律可循的,例如,用户可能每一个月发工资时购买得多,每年某个时间(双十一、生日)等购买得比较多马尔科夫模型能够挖掘出时间上的规律,假设我们能够根据用户上一次购买记录推测其下一次购买时间,就可以在推测时间向其发送邮件进行营销 至于营销的商品内容,可以根据其他推荐算法的结果。 输入:, , , ...ZSY40NYPS6,1381872876,2013-01-01,110...ZSY40NYPS6,1381872920,2013-01-11,32...ZSY40NYPS6,1381873821,2013-03-04,111...ZSY40NYPS6,1381874034,2013-04-09,65...第一步:生成 , , , , , , ...其中,purchaseDate1<=purchaseDate2<=purchaseDate3....需要对每个用户的交易日期排序,...ZSY40NYPS6,2013-01-01,110,2013-01-11,32,2013-03-04,111,2013-04-09,65... 第二步:将交易序列转换为状态序列对于每个用户的交易序列,每次取两个交易:2013-01-01,110,2013-01-11,32 2013-03-04,111,2013-04-09,65根据两个交易的时间差和金额差来标定不同的状态ZSY40NYPS6,ME,SL 第三步:生成马尔科夫状态转移矩阵。根据上步统计状态转移([ME,SL],1)累加 第四步:根据马尔科夫状态转移矩阵得到转移概率表 0.05033,0.008262,0.7487,0.1432,0.0003689,0.01423,0.03306,8.889e-05,0.0017910.4791,0.01265,0.4071,0.07468,0.0002040,0.009386,0.01612,0.0002040,0.00061210.6671,0.008839,0.1261,0.1463,0.0009289,0.01387,0.03505,0.0002426,0.0015550.04773,0.0004718,0.7681,0.01487,0.0001862,0.1385,0.02863,1.242e-05,0.0014900.6215,0.002151,0.2925,0.01075,0.006452,0.05161,0.008602,0.002151,0.0043010.1072,0.002772,0.7044,0.1364,0.0003616,0.01374,0.03247,8.036e-05,0.0026120.06196,0.0004748,0.7678,0.02008,0.0001424,0.1262,0.003988,4.748e-05,0.019370.5036,0.007299,0.3431,0.04380,0.007299,0.05839,0.007299,0.007299,0.021900.1834,0.001920,0.6313,0.02544,0.0004801,0.1167,0.03889,0.0009602,0.0009602 第五步:根据马尔科夫模型预测下一个智能邮件营销日期 import java.text.SimpleDateFormatimport org.apache.spark.SparkConfimport org.apache.spark.SparkContextobject Markov { def main(args: Array[String]): Unit = { val sparkConf = new SparkConf().setAppName("Markov").setMaster("local") val sc = new SparkContext(sparkConf) val input = "file:///media/chenjie/0009418200012FF3/ubuntu/java/Test1/input/smart_email_training.txt"; val output = "file:///media/chenjie/0009418200012FF3/ubuntu/markov" val transactions = sc.textFile(input) val dateFormat = new SimpleDateFormat("yyyy-MM-dd") val customers = transactions.map(line => { val tokens = line.split(",") (tokens(0), (dateFormat.parse(tokens(2)).getTime.toLong, tokens(3).toDouble)) //(用户ID,(交易日期,交易额)) }) val coustomerGrouped = customers.groupByKey()//将用户的所有交易聚合到一起 val sortedByDate = coustomerGrouped.mapValues(_.toList.sortBy(_._1))//将用户的所有交易按日期排序 val stateSequence = sortedByDate.mapValues(list => { val sequence = for { i <- 0 until list.size - 1 (currentDate, currentPurchase) = list(i) (nextDate, nextPurchse) = list(i + 1) } yield { val elapsedTime = (nextDate - currentDate) / 86400000 match { //1000ms/s * 60s/min * 60min/h * 24h/day = 86400000ms/day case diff if (diff < 30) => "S" //small case diff if (diff < 60) => "M" // medium case _ => "L" // large } val amountRange = (currentPurchase / nextPurchse) match { case ratio if (ratio < 0.9) => "L" // significantly less than case ratio if (ratio < 1.1) => "E" // more or less same case _ => "G" // significantly greater than } elapsedTime + amountRange } sequence }) val model = stateSequence.filter(_._2.size >= 2).flatMap(f => { val states = f._2 val statePair = for { i <- 0 until states.size - 1 fromState = states(i) toState = states(i + 1) } yield { ((fromState, toState), 1) } statePair }) val markovModel = model.reduceByKey(_ + _) val markovModelFormatted = markovModel.map(f => f._1._1 + "," + f._1._2 + "\t" + f._2) markovModelFormatted.foreach(println) markovModelFormatted.saveAsTextFile(output) sc.stop() }}
转载地址:http://qkqrb.baihongyu.com/