断电引起的mysql启动失败

发表于   |  

前两天实验室机房断电后发现服务器上的mysql启动不了了。通过查看日志（/var/log/mysqld.log）和网上查阅相关资料，发现是由于意外关机引起的权限crash，日志文件中mysql启动时报文件读取权限问题。然后参考帖子CentOS 6.5 - MySQL Daemon failed to start.修改了mysql的文件权限（/var/lib/mysql），具体权限是/var/lib/mysql的权限为755，/var/lib/mysql下的子目录及文件的权限为700，/var/lib/mysql/mysql下的子目录及文件权限为660。修改后的文件权限如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
755 /var/lib/mysql
660 /var/lib/mysql/mysqld-relay-bin.index
700 /var/lib/mysql/od@002dall
700 /var/lib/mysql/od@002dall/single_car_od.frm
700 /var/lib/mysql/od@002dall/db.opt
700 /var/lib/mysql/od@002dall/Statistic@0020of@0020the@0020seven@002dday@0020car@0020trajectories.frm
700 /var/lib/mysql/od@002dall/Statistic@0020of@0020the@0020seven@002dday@0020car@0020trajectories.MYI
700 /var/lib/mysql/od@002dall/eid.frm
700 /var/lib/mysql/od@002dall/eid.MYD
700 /var/lib/mysql/od@002dall/eid.MYI
700 /var/lib/mysql/od@002dall/Statistic@0020of@0020the@0020seven@002dday@0020car@0020trajectories.MYD
700 /var/lib/mysql/od@002dall/single_car_od.MYI
700 /var/lib/mysql/od@002dall/single_car_od.MYD
660 /var/lib/mysql/ibdata1
660 /var/lib/mysql/relay-log.info
660 /var/lib/mysql/mysqld-relay-bin.000001
660 /var/lib/mysql/mysql-bin.000233
660 /var/lib/mysql/mysql-bin.000231
660 /var/lib/mysql/mysql-bin.000232
700 /var/lib/mysql/test
660 /var/lib/mysql/test/testqjl.MYI
660 /var/lib/mysql/test/testqjl.MYD
660 /var/lib/mysql/test/testqjl.frm
700 /var/lib/mysql/mysql
660 /var/lib/mysql/mysql/time_zone_name.frm
660 /var/lib/mysql/mysql/event.MYD
660 /var/lib/mysql/mysql/tables_priv.frm
660 /var/lib/mysql/mysql/ndb_binlog_index.frm
660 /var/lib/mysql/mysql/help_topic.MYI
660 /var/lib/mysql/mysql/user.MYI
660 /var/lib/mysql/mysql/procs_priv.frm
660 /var/lib/mysql/mysql/time_zone.frm
660 /var/lib/mysql/mysql/time_zone.MYD
660 /var/lib/mysql/mysql/columns_priv.MYI
660 /var/lib/mysql/mysql/user.MYD
660 /var/lib/mysql/mysql/proc.MYD
660 /var/lib/mysql/mysql/servers.MYD
660 /var/lib/mysql/mysql/ndb_binlog_index.MYD
660 /var/lib/mysql/mysql/host.frm
660 /var/lib/mysql/mysql/func.frm
660 /var/lib/mysql/mysql/tables_priv.MYI
660 /var/lib/mysql/mysql/host.MYD
660 /var/lib/mysql/mysql/db.MYI
660 /var/lib/mysql/mysql/general_log.frm
660 /var/lib/mysql/mysql/time_zone_transition_type.MYI
660 /var/lib/mysql/mysql/procs_priv.MYD
660 /var/lib/mysql/mysql/help_relation.MYD
660 /var/lib/mysql/mysql/help_keyword.MYI
660 /var/lib/mysql/mysql/time_zone_leap_second.MYD
660 /var/lib/mysql/mysql/help_topic.frm
660 /var/lib/mysql/mysql/time_zone_transition.frm
660 /var/lib/mysql/mysql/slow_log.frm
660 /var/lib/mysql/mysql/help_keyword.frm
660 /var/lib/mysql/mysql/time_zone_name.MYD
660 /var/lib/mysql/mysql/plugin.MYI
660 /var/lib/mysql/mysql/help_relation.MYI
660 /var/lib/mysql/mysql/plugin.MYD
660 /var/lib/mysql/mysql/time_zone_leap_second.frm
660 /var/lib/mysql/mysql/ndb_binlog_index.MYI
660 /var/lib/mysql/mysql/columns_priv.frm
660 /var/lib/mysql/mysql/slow_log.CSM
660 /var/lib/mysql/mysql/func.MYI
660 /var/lib/mysql/mysql/help_topic.MYD
660 /var/lib/mysql/mysql/func.MYD
660 /var/lib/mysql/mysql/slow_log.CSV
660 /var/lib/mysql/mysql/help_keyword.MYD
660 /var/lib/mysql/mysql/user.frm
660 /var/lib/mysql/mysql/general_log.CSM
660 /var/lib/mysql/mysql/plugin.frm
660 /var/lib/mysql/mysql/time_zone_leap_second.MYI
660 /var/lib/mysql/mysql/help_category.MYI
660 /var/lib/mysql/mysql/servers.MYI
660 /var/lib/mysql/mysql/help_relation.frm
660 /var/lib/mysql/mysql/proc.frm
660 /var/lib/mysql/mysql/general_log.CSV
660 /var/lib/mysql/mysql/proc.MYI
660 /var/lib/mysql/mysql/time_zone_name.MYI
660 /var/lib/mysql/mysql/db.frm
660 /var/lib/mysql/mysql/columns_priv.MYD
660 /var/lib/mysql/mysql/time_zone_transition.MYD
660 /var/lib/mysql/mysql/host.MYI
660 /var/lib/mysql/mysql/help_category.frm
660 /var/lib/mysql/mysql/db.MYD
660 /var/lib/mysql/mysql/time_zone_transition_type.MYD
660 /var/lib/mysql/mysql/servers.frm
660 /var/lib/mysql/mysql/time_zone.MYI
660 /var/lib/mysql/mysql/time_zone_transition.MYI
660 /var/lib/mysql/mysql/time_zone_transition_type.frm
660 /var/lib/mysql/mysql/event.MYI
660 /var/lib/mysql/mysql/help_category.MYD
660 /var/lib/mysql/mysql/procs_priv.MYI
660 /var/lib/mysql/mysql/tables_priv.MYD
660 /var/lib/mysql/mysql/event.frm
700 /var/lib/mysql/od_week
700 /var/lib/mysql/od_week/single_car_od_200_lx.frm
700 /var/lib/mysql/od_week/single_car_od.frm
700 /var/lib/mysql/od_week/single_car_od_200_lx.MYI
700 /var/lib/mysql/od_week/countOD_cartype.frm
700 /var/lib/mysql/od_week/eid_200.MYI
700 /var/lib/mysql/od_week/single_car_od_200_diff_lx_has.MYI
700 /var/lib/mysql/od_week/rfid_ip_minute.frm
700 /var/lib/mysql/od_week/one_node_path_eid.frm
700 /var/lib/mysql/od_week/countOD_cartype.MYD
700 /var/lib/mysql/od_week/one_node_path_eid.MYD
700 /var/lib/mysql/od_week/single_car_od_copy1.MYI
700 /var/lib/mysql/od_week/rfid.MYI
700 /var/lib/mysql/od_week/single_car_od_copy1.MYD
700 /var/lib/mysql/od_week/eid_all_backup.MYD
700 /var/lib/mysql/od_week/rfid_ip_minute.MYD
700 /var/lib/mysql/od_week/eid_200.MYD
700 /var/lib/mysql/od_week/single_car_od_200_ccx.MYD
700 /var/lib/mysql/od_week/db.opt
700 /var/lib/mysql/od_week/kafka_eid_test.MYI
700 /var/lib/mysql/od_week/one_node_path_eid.MYI
700 /var/lib/mysql/od_week/rfid_ip_old.frm
700 /var/lib/mysql/od_week/single_car_od_200_diff_ccx_has.MYI
700 /var/lib/mysql/od_week/single_car_od_200_diff_lx_has.frm
700 /var/lib/mysql/od_week/single_car_od_200_ccx.frm
700 /var/lib/mysql/od_week/rfid_ip_60minute.frm
700 /var/lib/mysql/od_week/rfid_ip_old.MYI
700 /var/lib/mysql/od_week/kafka_eid_test.MYD
700 /var/lib/mysql/od_week/rfid.frm
700 /var/lib/mysql/od_week/rfid_ip_new.MYI
700 /var/lib/mysql/od_week/single_car_od_copy.MYD
700 /var/lib/mysql/od_week/rfid_ip_10minute.frm
700 /var/lib/mysql/od_week/single_car_od_200_diff_ccx_has.frm
700 /var/lib/mysql/od_week/rfid_ip_10minute.MYI
700 /var/lib/mysql/od_week/rfid_ip_new.frm
700 /var/lib/mysql/od_week/rfid_ip_10minute.MYD
700 /var/lib/mysql/od_week/single_car_od_200_diff_lx_has.MYD
700 /var/lib/mysql/od_week/rfid_ip_new.MYD
700 /var/lib/mysql/od_week/rfid_ip_minute.MYI
700 /var/lib/mysql/od_week/rfid_ip_60minute.MYD
700 /var/lib/mysql/od_week/single_car_od_copy1.frm
700 /var/lib/mysql/od_week/eid_all_backup.MYI
700 /var/lib/mysql/od_week/single_car_od_copy.MYI
700 /var/lib/mysql/od_week/countOD_cartype.MYI
700 /var/lib/mysql/od_week/eid.frm
700 /var/lib/mysql/od_week/single_car_od_200_lx.MYD
700 /var/lib/mysql/od_week/single_car_od_200_diff_ccx_has.MYD
700 /var/lib/mysql/od_week/eid.MYD
700 /var/lib/mysql/od_week/eid.MYI
700 /var/lib/mysql/od_week/eid_all_backup.frm
700 /var/lib/mysql/od_week/single_car_od.MYI
700 /var/lib/mysql/od_week/single_car_od_copy.frm
700 /var/lib/mysql/od_week/single_car_od_200_ccx.MYI
700 /var/lib/mysql/od_week/rfid_ip_new
700 /var/lib/mysql/od_week/rfid.MYD
700 /var/lib/mysql/od_week/rfid_ip_60minute.MYI
700 /var/lib/mysql/od_week/single_car_od.tar.gz
700 /var/lib/mysql/od_week/C:UserslxDesktopdiff.txt
700 /var/lib/mysql/od_week/rfid_ip_old.MYD
700 /var/lib/mysql/od_week/single_car_od.MYD
700 /var/lib/mysql/od_week/eid_200.frm
700 /var/lib/mysql/od_week/kafka_eid_test.frm
660 /var/lib/mysql/ib_logfile0
660 /var/lib/mysql/master.info
700 /var/lib/mysql/hi_db
700 /var/lib/mysql/hi_db/hi_tb.MYI
700 /var/lib/mysql/hi_db/db.opt
700 /var/lib/mysql/hi_db/hi_tb.frm
700 /var/lib/mysql/hi_db/hi_tb.MYD
660 /var/lib/mysql/ib_logfile1
700 /var/lib/mysql/o2odata
660 /var/lib/mysql/o2odata/offlinetrain.MYD
660 /var/lib/mysql/o2odata/User_Feature3.MYI
660 /var/lib/mysql/o2odata/offline_coupon_type_sum_per_merchant.MYD
660 /var/lib/mysql/o2odata/offline_popularityDegree.frm
660 /var/lib/mysql/o2odata/offlinetrain.frm
660 /var/lib/mysql/o2odata/offline_coupon_popularityDegree.MYD
660 /var/lib/mysql/o2odata/User_Feature8.MYD
660 /var/lib/mysql/o2odata/onlinetrain.MYI
660 /var/lib/mysql/o2odata/offline_consume_without_coupon.MYI
660 /var/lib/mysql/o2odata/offline_consume_without_coupon.frm
660 /var/lib/mysql/o2odata/offline_coupon_type_sum_per_merchant.MYI
660 /var/lib/mysql/o2odata/offlinetest.MYD
660 /var/lib/mysql/o2odata/offlinetrain.MYI
660 /var/lib/mysql/o2odata/User_Feature4.MYD
660 /var/lib/mysql/o2odata/offline_use_coupon.MYI
660 /var/lib/mysql/o2odata/offline_user_coupon_cross.frm
660 /var/lib/mysql/o2odata/offline_coupon_popularityDegree.MYI
660 /var/lib/mysql/o2odata/offline_coupon_sum_per_merchant.frm
660 /var/lib/mysql/o2odata/offline_consume_without_coupon.MYD
660 /var/lib/mysql/o2odata/User_Feature4.frm
660 /var/lib/mysql/o2odata/onlinetrain.frm
660 /var/lib/mysql/o2odata/offline_user_coupon_feature.MYD
660 /var/lib/mysql/o2odata/User_Feature2.frm
660 /var/lib/mysql/o2odata/db.opt
660 /var/lib/mysql/o2odata/offline_not_use_coupon.MYD
660 /var/lib/mysql/o2odata/offline_popularityDegree.MYI
660 /var/lib/mysql/o2odata/User_Feature1.MYI
660 /var/lib/mysql/o2odata/onlinetrain.MYD
660 /var/lib/mysql/o2odata/offline_coupon_popularityDegree.frm
660 /var/lib/mysql/o2odata/offline_user_coupon_feature.frm
660 /var/lib/mysql/o2odata/offlinetest.MYI
660 /var/lib/mysql/o2odata/offline_coupon_sum_per_merchant.MYI
660 /var/lib/mysql/o2odata/offline_user_coupon_cross.MYI
660 /var/lib/mysql/o2odata/User_Feature3.frm
660 /var/lib/mysql/o2odata/offline_use_coupon.MYD
660 /var/lib/mysql/o2odata/offlinetest.frm
660 /var/lib/mysql/o2odata/offline_not_use_coupon.frm
660 /var/lib/mysql/o2odata/User_Feature3.MYD
660 /var/lib/mysql/o2odata/User_Feature2.MYD
660 /var/lib/mysql/o2odata/offline_not_use_coupon.MYI
660 /var/lib/mysql/o2odata/offline_use_coupon.frm
660 /var/lib/mysql/o2odata/User_Feature2.MYI
660 /var/lib/mysql/o2odata/offline_user_has_coupon.MYD
660 /var/lib/mysql/o2odata/offline_user_coupon_feature.MYI
660 /var/lib/mysql/o2odata/User_Feature8.MYI
660 /var/lib/mysql/o2odata/User_Feature4.MYI
660 /var/lib/mysql/o2odata/offline_popularityDegree.MYD
660 /var/lib/mysql/o2odata/offline_coupon_sum_per_merchant.MYD
660 /var/lib/mysql/o2odata/offline_user_has_coupon.frm
660 /var/lib/mysql/o2odata/offline_coupon_type_sum_per_merchant.frm
660 /var/lib/mysql/o2odata/offline_user_coupon_cross.MYD
660 /var/lib/mysql/o2odata/offline_user_has_coupon.MYI
660 /var/lib/mysql/o2odata/User_Feature1.MYD
660 /var/lib/mysql/o2odata/User_Feature1.frm
660 /var/lib/mysql/o2odata/User_Feature8.frm
600 /var/lib/mysql/.bash_history
777 /var/lib/mysql/mysql.sock
600 /var/lib/mysql/.mysql_history
660 /var/lib/mysql/mysql-bin.index
660 /var/lib/mysql/mysqld-relay-bin.000002

但还是启动失败，但文件权限问题解决了，具体报错如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
161019 15:44:18 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
161019 15:44:18  InnoDB: Initializing buffer pool, size = 8.0M
161019 15:44:18  InnoDB: Completed initialization of buffer pool
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
161019 15:44:18  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Page directory corruption: infimum not pointed to
161019 15:44:18  InnoDB: Page dump in ascii and hex (16384 bytes):
....
161019 15:44:18  InnoDB: Page checksum 1575996416, prior-to-4.0.14-form checksum 1371122432
InnoDB: stored checksum 0, prior-to-4.0.14-form stored checksum 0
InnoDB: Page lsn 0 0, low 4 bytes of lsn at page end 0
InnoDB: Page number (if stored to page already) 0,
InnoDB: space id (if created with >= MySQL-4.1.1 and stored already) 0
InnoDB: Page may be a freshly allocated page
InnoDB: Page directory corruption: supremum not pointed to
161019 15:44:18  InnoDB: Page dump in ascii and hex (16384 bytes):

查看网上分析是因为mysql日志已经损坏，这里采用非常规手段，修改innodb_force_recovery参数，使mysqld跳过恢复步骤。innodb_force_recovery参数的值可以设为0～6，0为默认的正常启动值，可在文件/etc/my.cnf文件中设置，关于innodb_force_recovery可查看文档Forcing InnoDB Recovery。但强制启动还是没启动起来，在这里折腾了很久，也没有看到有类似的问题。最后发现删除了mysql的日志文件后重启成功了！具体的文件为/var/lib/mysql目录下的ib_logfile0、ib_logfile1文件，删除后就启动成功了。
特记录下，希望对遇到同样问题的人有所帮助。