如何在不使用BOM且以非ASCII字符开头的情况下针对文件识别不同的编码?
|
尝试识别不带BOM的文件的编码时出现问题,尤其是当文件以非ASCII字符开头时。
我发现了以下两个有关如何识别文件编码的主题,
在不使用BOM的情况下如何识别不同的编码?
Java:阅读器和编码
目前,我创建了一个类来标识文件的不同编码(例如UTF-8,UTF-16,UTF-32,UTF-16 no BOM等),如下所示,
public class UnicodeReader extends Reader {
private static final int BOM_SIZE = 4;
private final InputStreamReader reader;
/**
* Construct UnicodeReader
* @param in Input stream.
* @param defaultEncoding Default encoding to be used if BOM is not found,
* or <code>null</code> to use system default encoding.
* @throws IOException If an I/O error occurs.
*/
public UnicodeReader(InputStream in, String defaultEncoding) throws IOException {
byte bom[] = new byte[BOM_SIZE];
String encoding;
int unread;
PushbackInputStream pushbackStream = new PushbackInputStream(in, BOM_SIZE);
int n = pushbackStream.read(bom, 0, bom.length);
// Read ahead four bytes and check for BOM marks.
if ((bom[0] == (byte) 0xEF) && (bom[1] == (byte) 0xBB) && (bom[2] == (byte) 0xBF)) {
encoding = \"UTF-8\";
unread = n - 3;
} else if ((bom[0] == (byte) 0xFE) && (bom[1] == (byte) 0xFF)) {
encoding = \"UTF-16BE\";
unread = n - 2;
} else if ((bom[0] == (byte) 0xFF) && (bom[1] == (byte) 0xFE)) {
encoding = \"UTF-16LE\";
unread = n - 2;
} else if ((bom[0] == (byte) 0x00) && (bom[1] == (byte) 0x00) && (bom[2] == (byte) 0xFE) && (bom[3] == (byte) 0xFF)) {
encoding = \"UTF-32BE\";
unread = n - 4;
} else if ((bom[0] == (byte) 0xFF) && (bom[1] == (byte) 0xFE) && (bom[2] == (byte) 0x00) && (bom[3] == (byte) 0x00)) {
encoding = \"UTF-32LE\";
unread = n - 4;
} else {
// No BOM detected but still could be UTF-16
int found = 0;
for (int i = 0; i < 4; i++) {
if (bom[i] == (byte) 0x00)
found++;
}
if(found >= 2) {
if(bom[0] == (byte) 0x00){
encoding = \"UTF-16BE\";
}
else {
encoding = \"UTF-16LE\";
}
unread = n;
}
else {
encoding = defaultEncoding;
unread = n;
}
}
// Unread bytes if necessary and skip BOM marks.
if (unread > 0) {
pushbackStream.unread(bom, (n - unread), unread);
} else if (unread < -1) {
pushbackStream.unread(bom, 0, 0);
}
// Use given encoding.
if (encoding == null) {
reader = new InputStreamReader(pushbackStream);
} else {
reader = new InputStreamReader(pushbackStream, encoding);
}
}
public String getEncoding() {
return reader.getEncoding();
}
public int read(char[] cbuf, int off, int len) throws IOException {
return reader.read(cbuf, off, len);
}
public void close() throws IOException {
reader.close();
}
}
上面的代码可以在所有情况下正常工作,除非文件没有BOM且以非ASCII字符开头。由于在这种情况下,用于检查文件是否仍为没有BOM的UTF-16的逻辑将无法正常工作,并且默认会将编码设置为UTF-8。
是否有一种方法可以检查没有BOM的文件的编码并以非ASCII字符开头,尤其是对于UTF-16 NO BOM文件?
谢谢,任何想法将不胜感激。
没有找到相关结果
已邀请:
3 个回复
磁辫覆氓
期差骇蓟
郡豪靠暖