ORA-01033: ORACLE initialization or shutdown in progress—惜分飞

客户反馈数据库使用plsql dev登录报ORA-01033: ORACLE initialization or shutdown in progress的错误
 

20240122211338


出现该错误一般是由于数据库没有正常open成功,查看oracle 告警日志发现

Mon Jan 22 16:55:50 2024

Database mounted in Exclusive Mode

Lost write protection disabled

Completed: alter database mount exclusive

alter database open

Beginning crash recovery of 1 threads

 parallel recovery started with 15 processes

Started redo scan

Completed redo scan

 read 139 KB redo, 70 data blocks need recovery

Errors in file d:appadministratordiag
dbmsorclorcl raceorcl_ora_7792.trc  (incident=20565):

ORA-00600: ??????, ??: [kcratr_nab_less_than_odr], [1], [1916], [28210], [28222], [], [], [], [], [], [], []

Incident details in: d:appadministratordiag
dbmsorclorclincidentincdir_20565orcl_ora_7792_i20565.trc

Mon Jan 22 16:55:57 2024

Trace dumping is performing id=[cdmp_20240122165557]

Aborting crash recovery due to error 600

Errors in file d:appadministratordiag
dbmsorclorcl raceorcl_ora_7792.trc:

ORA-00600: ??????, ??: [kcratr_nab_less_than_odr], [1], [1916], [28210], [28222], [], [], [], [], [], [], []

Errors in file d:appadministratordiag
dbmsorclorcl raceorcl_ora_7792.trc:

ORA-00600: ??????, ??: [kcratr_nab_less_than_odr], [1], [1916], [28210], [28222], [], [], [], [], [], [], []

ORA-600 signalled during: alter database open...

这个错误比较常见,解决起来比较简单,参考:
kcratr_nab_less_than_odr
12c启动报kcratr_nab_less_than_odr
又一例ORA-600 kcratr_nab_less_than_odr
在恢复过程中中还遇到了ORA-00700 kcrf_split_brain_error错误,但是没有影响数据库open

Mon Jan 22 20:13:55 2024

alter database open

Beginning crash recovery of 1 threads

Started redo application at

 Thread 1: logseq 1916, block 27931

Recovery of Online Redo Log: Thread 1 Group 2 Seq 1916 Reading mem 0

  Mem# 0: D:TEMPORCLREDO02.LOG

Completed redo application of 0.00MB

Completed crash recovery at

 Thread 1: logseq 1916, block 28210, scn 43957072

 0 data blocks read, 0 data blocks written, 139 redo k-bytes read

Errors in file d:appxifenfeidiag
dbmsorclorcl raceorcl_ora_6104.trc  (incident=15729):

ORA-00700: 软内部错误, 参数: [kcrf_split_brain_error], [1], [1916], [28222], [28209], [4], [], [], [], [], [], []

Incident details in: d:appxifenfeidiag
dbmsorclorclincidentincdir_15729orcl_ora_6104_i15729.trc

Mon Jan 22 20:13:56 2024

Trace dumping is performing id=[cdmp_20240122201356]

Mon Jan 22 20:13:56 2024

Thread 1 advanced to log sequence 1917 (thread open)

Thread 1 opened at log sequence 1917

  Current log# 3 seq# 1917 mem# 0: D:TEMPORCLREDO03.LOG

Successful open of redo thread 1

Mon Jan 22 20:13:56 2024

SMON: enabling cache recovery

Verifying file header compatibility for 11g tablespace encryption..

Verifying 11g file header compatibility for tablespace encryption completed

SMON: enabling tx recovery

Database Characterset is AL32UTF8

replication_dependency_tracking turned off (no async multimaster replication found)

WARNING: AQ_TM_PROCESSES is set to 0. System operation                     might be adversely affected.

Completed: alter database open

20240122201556


至此数据库open成功但是dbv检测system有很多坏块需要分析处理

C:UsersXIFENFEI>dbv file=d:/temp/orcl/system01.dbf

DBVERIFY: Release 11.2.0.1.0 - Production on 星期一 1月 22 21:07:18 2024

Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.

DBVERIFY - 开始验证: FILE = D:TEMPORCLSYSTEM01.DBF

页 106278 流入 - 很可能是介质损坏

Corrupt block relative dba: 0x00419f26 (file 1, block 106278)

Fractured block found during dbv:

Data in bad block:

 type: 6 format: 2 rdba: 0x00419f26

 last change scn: 0x0000.01410f78 seq: 0x2 flg: 0x04

 spare1: 0x0 spare2: 0x0 spare3: 0x0

 consistency value in tail: 0x00000000

 check value in block header: 0xbf11

 computed block checksum: 0xaf18

页 106279 标记为损坏

Corrupt block relative dba: 0x00419f27 (file 1, block 106279)

Bad header found during dbv:

Data in bad block:

 type: 178 format: 3 rdba: 0xc8c9c1b6

 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00

 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf

 consistency value in tail: 0x00000000

 check value in block header: 0xbbb2

 block checksum disabled

页 106280 标记为损坏

Corrupt block relative dba: 0x00419f28 (file 1, block 106280)

Bad header found during dbv:

Data in bad block:

 type: 178 format: 3 rdba: 0xc8c9c1b6

 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00

 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf

 consistency value in tail: 0x00000000

 check value in block header: 0xbbb2

 block checksum disabled

页 106281 标记为损坏

Corrupt block relative dba: 0x00419f29 (file 1, block 106281)

Bad header found during dbv:

Data in bad block:

 type: 178 format: 3 rdba: 0xc8c9c1b6

 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00

 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf

 consistency value in tail: 0x00000000

 check value in block header: 0xbbb2

 block checksum disabled

页 106282 标记为损坏

Corrupt block relative dba: 0x00419f2a (file 1, block 106282)

Bad header found during dbv:

Data in bad block:

 type: 178 format: 3 rdba: 0xc8c9c1b6

 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00

 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf

 consistency value in tail: 0x00000000

 check value in block header: 0xbbb2

 block checksum disabled

页 106283 标记为损坏

Corrupt block relative dba: 0x00419f2b (file 1, block 106283)

Bad header found during dbv:

Data in bad block:

 type: 178 format: 3 rdba: 0xc8c9c1b6

 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00

 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf

 consistency value in tail: 0x00000000

 check value in block header: 0xbbb2

 block checksum disabled

页 106284 标记为损坏

Corrupt block relative dba: 0x00419f2c (file 1, block 106284)

Bad header found during dbv:

Data in bad block:

 type: 178 format: 3 rdba: 0xc8c9c1b6

 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00

 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf

 consistency value in tail: 0x00000000

 check value in block header: 0xbbb2

 block checksum disabled

页 106285 标记为损坏

Corrupt block relative dba: 0x00419f2d (file 1, block 106285)

Bad header found during dbv:

Data in bad block:

 type: 178 format: 3 rdba: 0xc8c9c1b6

 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00

 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf

 consistency value in tail: 0x00000000

 check value in block header: 0xbbb2

 block checksum disabled

页 106286 标记为损坏

Corrupt block relative dba: 0x00419f2e (file 1, block 106286)

Bad header found during dbv:

Data in bad block:

 type: 178 format: 3 rdba: 0xc8c9c1b6

 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00

 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf

 consistency value in tail: 0x2c310602

 check value in block header: 0xbbb2

 block checksum disabled

页 143094 流入 - 很可能是介质损坏

Corrupt block relative dba: 0x00422ef6 (file 1, block 143094)

Fractured block found during dbv:

Data in bad block:

 type: 6 format: 2 rdba: 0x00422ef6

 last change scn: 0x0000.028f863b seq: 0x2 flg: 0x04

 spare1: 0x0 spare2: 0x0 spare3: 0x0

 consistency value in tail: 0x00000000

 check value in block header: 0xda23

 computed block checksum: 0x4210

页 143095 标记为损坏

Corrupt block relative dba: 0x00422ef7 (file 1, block 143095)

Bad header found during dbv:

Data in bad block:

 type: 178 format: 3 rdba: 0xc8c9c1b6

 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00

 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf

 consistency value in tail: 0x00000000

 check value in block header: 0xbbb2

 block checksum disabled

页 143096 标记为损坏

Corrupt block relative dba: 0x00422ef8 (file 1, block 143096)

Bad header found during dbv:

Data in bad block:

 type: 178 format: 3 rdba: 0xc8c9c1b6

 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00

 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf

 consistency value in tail: 0x00000000

 check value in block header: 0xbbb2

 block checksum disabled

页 143097 标记为损坏

Corrupt block relative dba: 0x00422ef9 (file 1, block 143097)

Bad header found during dbv:

Data in bad block:

 type: 178 format: 3 rdba: 0xc8c9c1b6

 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00

 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf

 consistency value in tail: 0x00000000

 check value in block header: 0xbbb2

 block checksum disabled

页 143098 标记为损坏

Corrupt block relative dba: 0x00422efa (file 1, block 143098)

Bad header found during dbv:

Data in bad block:

 type: 178 format: 3 rdba: 0xc8c9c1b6

 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00

 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf

 consistency value in tail: 0x00000000

 check value in block header: 0xbbb2

 block checksum disabled

页 143099 标记为损坏

Corrupt block relative dba: 0x00422efb (file 1, block 143099)

Bad header found during dbv:

Data in bad block:

 type: 178 format: 3 rdba: 0xc8c9c1b6

 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00

 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf

 consistency value in tail: 0x00000000

 check value in block header: 0xbbb2

 block checksum disabled

页 143100 标记为损坏

Corrupt block relative dba: 0x00422efc (file 1, block 143100)

Bad header found during dbv:

Data in bad block:

 type: 178 format: 3 rdba: 0xc8c9c1b6

 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00

 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf

 consistency value in tail: 0x00000000

 check value in block header: 0xbbb2

 block checksum disabled

页 143101 标记为损坏

Corrupt block relative dba: 0x00422efd (file 1, block 143101)

Bad header found during dbv:

Data in bad block:

 type: 178 format: 3 rdba: 0xc8c9c1b6

 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00

 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf

 consistency value in tail: 0x00000000

 check value in block header: 0xbbb2

 block checksum disabled

页 143102 标记为损坏

Corrupt block relative dba: 0x00422efe (file 1, block 143102)

Bad header found during dbv:

Data in bad block:

 type: 178 format: 3 rdba: 0xc8c9c1b6

 last change scn: 0x0000.0a0df8c7 seq: 0x0 flg: 0x00

 spare1: 0xbf spare2: 0xc9 spare3: 0xc9bf

 consistency value in tail: 0x8a780602

 check value in block header: 0xbbb2

 block checksum disabled

DBVERIFY - 验证完成

检查的页总数: 152320

处理的页总数 (数据): 115189

失败的页总数 (数据): 0

处理的页总数 (索引): 13086

失败的页总数 (索引): 0

处理的页总数 (其他): 9741

处理的总页数 (段)  : 1

失败的总页数 (段)  : 0

空的页总数: 14286

标记为损坏的总页数: 18

流入的页总数: 2

加密的总页数        : 0

最高块 SCN            : 44036082 (0.44036082)

通过分析aud$的extent,确认这些坏块全部属于该对象

SQL> select block_id,blocks from dba_extents where segment_name='AUD$';

  BLOCK_ID     BLOCKS

---------- ----------

…………

    102016       1024

    103040       1024

    104064       1024

    105088       1024

    106112       1024

…………

    141056       1024

    142080       1024

    143104       1024

已选择124行。

处理方法比较简单,直接truncate aud$表即可