分类 elasticsearch 下的文章

logstash 从 MySQL 动态同步日志数据到 ElasticSearch

作者: admin

时间: 2019-11-26

分类: elasticsearch

- 阅读剩余部分 -

记一次logstash-input-jdbc配置文件type冲突血泪史

作者: admin

时间: 2018-06-26

分类: elasticsearch

# 记一次logstash-input-jdbc配置文件type冲突血泪史

### 起因

统计需求，需要将某几份数据按时间分组进行统计分析，使用MySQL进行分组查询，不只SQL语句繁琐，且效率较慢，按时间分组查询SQL如下：

```sql
-- month
select CONCAT(YEAR(stat_date),'_',DATE_FORMAT(stat_date,'%m')) months ,sum(total_amount) as count_amount, sum(total_new_user) as count_new_user, sum(da_active_user) as count_active_user from xxx
WHERE `stat_date` BETWEEN '2016-01-02' AND '2017-05-30' group by months;

-- 季度
select CONCAT(YEAR(stat_date),'_',quarter(stat_date)) qu,sum(total_amount) as count_amount, sum(total_new_user) as count_new_user, sum(da_active_user) as count_active_user from xxx
WHERE `stat_date` BETWEEN '2016-01-02' AND '2017-05-30' group by qu;

-- 周
select CONCAT(YEAR(stat_date),'_',DATE_FORMAT(stat_date,'%U')) weeks,sum(total_amount) as count_amount, sum(total_new_user) as count_new_user, sum(da_active_user) as count_active_user from xxx
WHERE `stat_date` BETWEEN '2016-01-02' AND '2017-05-30' group by weeks;

-- 天
select CONCAT(YEAR(stat_date),'_',DATE_FORMAT(stat_date,'%m'),'_',DATE_FORMAT(stat_date,'%d')) days, sum(total_amount) as count_amount, sum(total_new_user) as count_new_user, sum(da_active_user) as count_active_user from xxx
WHERE `stat_date` BETWEEN '2016-01-02' AND '2017-05-30' group by days;
```

- 阅读剩余部分 -