前言
在测试过程中,有的时候登录需要输入图片验证码。这时候使用Selenium进行自动化测试,怎么做图片验证码识别?本篇内容主要介绍使用Selenium、BufferedImage、Tesseract进行图片 验证码识别。
环境准备
jdk:1.8
tessdata:文章末尾附下载地址
安装Tesseract
我本地是ubuntu系统
sudo apt install tesseract-ocr
sudo apt install libtesseract-dev
在项目中引用
<dependency><groupId>net.sourceforge.tess4j</groupId><artifactId>tess4j</artifactId><version>4.5.4</version>
</dependency>
实现
在下图中,登录需要使用图片验证码进行验证。我们的图片验证码识别流程是使用Selenium定位到图片验证码元素,将元素截图保。然后将保存的图片验证码使用BufferedImage进行灰度化、二值化处理,处理完成后去除图片上的干扰点。最后使用Tesseract进行图片验证码上的字符识别。
处理图片
首先使用BufferedImage读取图片验证码图片,然后调整亮度后进行灰度化、二值化处理。处理后的图片去除干扰点。
public static void cleanLinesInImage(File sfile, String destDir) throws IOException{File destF =new File(destDir);if (!destF.exists()){destF.mkdirs();}BufferedImage bufferedImage = ImageIO.read(sfile);int h = bufferedImage.getHeight();int w = bufferedImage.getWidth();// 灰度化int[][] gray = new int[w][h];for (int x = 0; x < w; x++){for (int y = 0; y < h; y++){int argb = bufferedImage.getRGB(x, y);// 图像加亮(调整亮度识别率非常高)int r = (int) (((argb >> 16) & 0xFF) * 1.1 + 30);int g = (int) (((argb >> 8) & 0xFF) * 1.1 + 30);int b = (int) (((argb >> 0) & 0xFF) * 1.1 + 30);// int r = (int) (((argb >> 16) & 0xFF) * 0.1 + 30);// int g = (int) (((argb >> 8) & 0xFF) * 0.1 + 30);// int b = (int) (((argb >> 0) & 0xFF) * 0.1 + 30);if (r >= 255){r = 255;}if (g >= 255){g = 255;}if (b >= 255){b = 255;}gray[x][y] = (int) Math.pow((Math.pow(r, 2.2) * 0.2973 + Math.pow(g, 2.2)* 0.6274 + Math.pow(b, 2.2) * 0.0753), 1 / 2.2);}}ImageIO.write(bufferedImage, "jpg", new File(destDir, sfile.getName()));// 二值化int threshold = ostu(gray, w, h);BufferedImage binaryBufferedImage = new BufferedImage(w, h, BufferedImage.TYPE_BYTE_BINARY);for (int x = 0; x < w; x++){for (int y = 0; y < h; y++){if (gray[x][y] > threshold){gray[x][y] |= 0x00FFFF;} else{gray[x][y] &= 0xFF0000;}binaryBufferedImage.setRGB(x, y, gray[x][y]);}}ImageIO.write(binaryBufferedImage, "jpg", new File(destDir, sfile.getName()));// 去除干扰线条for(int y = 1; y < h-1; y++){for(int x = 1; x < w-1; x++){boolean flag = false ;if(isBlack(binaryBufferedImage.getRGB(x, y))){//左右均为空时,去掉此点if(isWhite(binaryBufferedImage.getRGB(x-1, y)) && isWhite(binaryBufferedImage.getRGB(x+1, y))){flag = true;}//上下均为空时,去掉此点if(isWhite(binaryBufferedImage.getRGB(x, y+1)) && isWhite(binaryBufferedImage.getRGB(x, y-1))){flag = true;}//斜上下为空时,去掉此点if(isWhite(binaryBufferedImage.getRGB(x-1, y+1)) && isWhite(binaryBufferedImage.getRGB(x+1, y-1))){flag = true;}if(isWhite(binaryBufferedImage.getRGB(x+1, y+1)) && isWhite(binaryBufferedImage.getRGB(x-1, y-1))){flag = true;}if(flag){binaryBufferedImage.setRGB(x,y,-1);}}}}// 矩阵打印// for (int y = 0; y < h; y++)// {// for (int x = 0; x < w; x++)// {// if (isBlack(binaryBufferedImage.getRGB(x, y)))// {// System.out.print("*");// } else// {// System.out.print(" ");// }// }// System.out.println();// }ImageIO.write(binaryBufferedImage, "jpg", new File(destDir, sfile.getName()));}
OCR识别
setDataPath方法,传入你下载的
public static String executeTess4J(String imgUrl){String ocrResult = "";try{ITesseract instance = new Tesseract();instance.setDatapath("your tessdata path");instance.setLanguage("eng");instance.setOcrEngineMode(0);instance.setTessVariable("tessedit_char_whitelist", "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz01234567890");File imgDir = new File(imgUrl);//long startTime = System.currentTimeMillis();ocrResult = instance.doOCR(imgDir);}catch (TesseractException e){e.printStackTrace();}return ocrResult;
}
验证
编写Selenium脚本
public static void main(String[] args) throws IOException {System.setProperty("webdriver.chrome.driver", "/home/zhangkexin/chromedriver");WebDriver driver = new ChromeDriver();driver.manage().window().maximize();driver.manage().timeouts().implicitlyWait(20, TimeUnit.SECONDS);driver.get("https://xkczb.jtw.beijing.gov.cn/#");WebElement element = driver.findElement(By.xpath("//*[@id=\"getValidCode\"]/img"));File img = element.getScreenshotAs(OutputType.FILE);String path = System.getProperty("user.dir");cleanLinesInImage(img, path);String imgFile = path + "/" + img.getName();Path source = Paths.get(imgFile);Path dest = Paths.get("/home/zhangkexin/ui-test/autoTest/img.jpg");Files.copy(source, dest, StandardCopyOption.REPLACE_EXISTING);String code = executeTess4J("/home/zhangkexin/ui-test/autoTest/img.jpg");System.out.println(code);driver.quit();
}
看一下经过处理后的图片验证码
最后实际识别出来的结果。
testdata:
链接:https://pan.baidu.com/s/1uJE9wl1oa2WAsBTsydUlmg?pwd=m576
提取码:m576